-- Logs begin at Mon 2022-05-23 14:38:19 UTC, end at Mon 2022-05-23 16:35:49 UTC. -- May 23 15:35:17.205664 dv-cinder-835881-14-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 23 15:35:17.214096 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 15:35:17 2022] *** May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: nodename: dv-cinder-835881-14-iscsi May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: machine: x86_64 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: clock source: unix May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: pcre jit disabled May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: detected number of CPU cores: 4 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: current working directory: / May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: detected binary path: /usr/bin/uwsgi-core May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: your processes number limit is 47680 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: your memory page size is 4096 bytes May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: detected max file descriptor number: 2048 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: lock engine: pthread robust mutexes May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: thunder lock: enabled May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 23 15:35:17.218260 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 15:35:17.263116 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: Python main interpreter initialized at 0x560f85fa09c0 May 23 15:35:17.263116 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: python threads support enabled May 23 15:35:17.263116 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: your server socket listen backlog is limited to 100 connections May 23 15:35:17.263116 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: your mercy for graceful operations on workers is 90 seconds May 23 15:35:17.263509 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: mapped 403077 bytes (393 KB) for 2 cores May 23 15:35:17.263609 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: *** Operational MODE: preforking *** May 23 15:35:17.263824 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: *** uWSGI is running in multiple interpreter mode *** May 23 15:35:17.263824 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: spawned uWSGI master process (pid: 110500) May 23 15:35:17.264054 dv-cinder-835881-14-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 23 15:35:17.265033 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: spawned uWSGI worker 1 (pid: 110501, cores: 1) May 23 15:35:17.265033 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: spawned uWSGI worker 2 (pid: 110502, cores: 1) May 23 15:35:17.265179 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 15:35:18.449974 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 15:35:18.450700 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 15:35:19.182651 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:19.182651 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: from cryptography.utils import int_from_bytes May 23 15:35:19.182986 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:19.182986 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: from cryptography.utils import int_from_bytes May 23 15:35:19.189960 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:19.189960 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: from cryptography.utils import int_from_bytes May 23 15:35:19.190289 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:19.190289 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: from cryptography.utils import int_from_bytes May 23 15:35:21.837273 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:35:21.837962 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:35:21.838061 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:35:21.838395 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:35:21.838642 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:35:21.839032 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.839369 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.839683 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.839996 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.840317 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.840621 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.840935 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.841248 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.841570 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.841827 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.842139 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.842462 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835881-14-iscsi {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.842779 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.843081 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.843223 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:35:21.843633 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.843711 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:35:21.843783 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.844015 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:35:21.844216 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.844319 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.844445 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:35:21.844616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.844731 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:35:21.845110 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.845213 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.845434 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.845560 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.845753 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.845866 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846071 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846194 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846506 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846592 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846718 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.846808 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847000 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847089 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847368 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847463 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847672 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847757 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.847943 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848036 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848258 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848407 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848586 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835881-14-iscsi {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848685 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835881-14-iscsi {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848908 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.848993 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849216 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849303 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849526 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849618 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849845 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.849946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850223 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850325 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850547 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850647 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850853 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.850946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.851216 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.851403 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.851504 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.851713 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.851828 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852017 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852146 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852328 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852475 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852660 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852782 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.852973 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853229 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853369 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853490 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853677 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853813 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.853931 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854066 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854183 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854316 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854480 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854612 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854746 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.854880 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835881-14-iscsi {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855048 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855215 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855394 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855516 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855745 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.855887 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856070 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856288 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856457 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856614 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856768 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.856922 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857079 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857217 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857371 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857509 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857670 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857799 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.857897 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.858104 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.858263 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.858422 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.858741 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.858843 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.859060 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.859359 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.859447 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.859689 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.859942 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.860045 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.860761 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.861254 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.861613 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.861877 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862041 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862213 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862370 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862517 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862662 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862819 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.862966 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.863158 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.863334 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.863496 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.863641 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.32 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.863789 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.32 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864022 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864022 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864307 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864444 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864620 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864773 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.864933 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865084 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865229 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865387 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865527 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865680 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865821 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.865980 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866122 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866275 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866430 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866586 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866732 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.866884 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867027 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867215 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867373 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867544 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867686 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867827 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.867972 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868115 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868258 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868414 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868581 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868796 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.868951 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869108 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869217 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869307 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.32 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869387 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869465 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.32 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869542 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869625 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869712 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.869924 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870012 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870242 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870329 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870549 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870644 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870847 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.870942 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871170 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871260 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871569 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871775 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.871860 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.872075 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.872160 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.872378 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.872473 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874406 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874535 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874632 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874713 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874857 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.874947 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875035 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875260 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875353 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875443 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875522 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875602 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875679 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875757 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875843 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.875930 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876010 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876095 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876174 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876251 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876328 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876417 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876504 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876583 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876667 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876745 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876829 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.876921 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877001 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877079 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877155 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877365 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877464 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877662 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877748 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.877975 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.878069 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.878276 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.878379 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.878576 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.878667 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879020 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879138 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879350 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879523 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879657 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.879963 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880056 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880288 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880431 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880582 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880713 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.880877 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.881001 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.881173 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.881804 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882028 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882175 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882413 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882569 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882746 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.882876 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.883051 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.883199 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.883635 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.883813 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.883980 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.884107 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.884414 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.884520 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.884612 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.884694 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.884915 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.885017 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.885226 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.885313 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.885532 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.885626 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.885839 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.885940 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.886145 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.886244 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.886450 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.886536 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.886754 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.886848 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.887058 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.887166 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.887604 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.887792 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.887908 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.887992 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.888070 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.888148 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.888300 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.888395 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.888609 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.888697 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.888913 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.888994 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.889217 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.889303 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.889535 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:21.889623 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.889848 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.889940 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890155 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890251 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890470 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890558 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890796 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.890881 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891135 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891239 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891445 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891557 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891784 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.891873 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892092 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892178 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892405 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892493 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892730 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.892815 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893044 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893137 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893351 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893443 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893660 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893745 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.893968 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894147 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894305 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894393 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894601 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894751 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.894918 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895052 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895255 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895397 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895575 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895706 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.895887 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896016 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896203 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896324 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896530 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896620 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.896868 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.897027 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.897719 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.897809 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898064 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898150 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898422 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898532 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898712 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.898799 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899026 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899133 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899358 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899447 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899666 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899754 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.899974 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900061 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900275 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900369 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900578 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900666 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.900966 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901182 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901270 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901518 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901631 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901797 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.901894 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902207 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902423 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902509 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902757 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.902926 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903065 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903236 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903393 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903481 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903701 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.903802 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904006 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904100 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904397 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904613 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904708 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.904919 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905015 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905220 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905319 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905540 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905628 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905848 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.905988 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906143 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906230 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906444 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906531 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906748 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.906833 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907051 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907235 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907370 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907457 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907681 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907801 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.907984 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908111 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908294 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908431 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908595 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908742 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.908914 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909056 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909223 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909370 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909531 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909683 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909859 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.909989 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910184 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910300 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910483 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910610 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910801 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.910901 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911138 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911251 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911452 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911563 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911751 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.911869 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912055 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912354 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912518 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912774 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.912862 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913077 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913171 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913392 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913553 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913682 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913767 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.913972 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914068 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914272 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914365 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914569 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914657 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914866 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.914962 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915209 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915514 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915601 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915811 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.915933 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916108 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916240 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916406 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916563 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916744 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.916887 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917044 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917204 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917380 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917508 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917710 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.917821 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918024 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918119 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918343 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918430 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918650 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918736 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.918958 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919058 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919287 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919375 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919593 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919679 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919894 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.919991 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920197 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920283 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920497 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920583 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920803 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.920889 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921112 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921206 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921414 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921500 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921716 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.921804 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922017 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922103 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922329 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922412 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922629 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922716 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.922938 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923030 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923299 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923400 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923630 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923717 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.923939 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924024 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924249 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924344 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924553 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924638 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924859 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.924952 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925161 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925254 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925468 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925558 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925774 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.925859 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926075 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926168 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926386 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926471 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926692 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926777 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.926997 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927322 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927408 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927638 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927731 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.927943 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.928036 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.928756 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.928844 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929055 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929140 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929363 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929450 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929783 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.929920 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930012 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930094 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930304 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930393 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930609 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930709 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.930924 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931047 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931256 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931346 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931561 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931658 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.931862 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932036 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932161 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932256 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932448 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932536 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932751 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.932855 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933063 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933181 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933367 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933527 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933673 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933775 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.933971 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934095 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934276 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934371 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934575 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934669 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934871 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.934956 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935191 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935297 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935499 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935595 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935816 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.935910 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936250 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936437 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936571 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936753 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.936879 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937052 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937178 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937452 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937540 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937777 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.937861 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938081 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938211 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938391 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938479 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938703 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.938789 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939011 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939144 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939360 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939468 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939667 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939770 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.939960 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940063 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940286 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940399 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940568 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940664 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.940916 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.941069 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.941163 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.941375 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.941779 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.941964 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942057 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942148 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942230 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942494 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942576 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942644 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942805 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.942941 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.943157 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.943272 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.943462 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.943560 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.943816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.944461 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.944621 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.944786 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.944926 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945095 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945230 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945403 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945573 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945712 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.945881 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946013 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946184 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946319 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946499 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946632 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.946817 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947185 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947303 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947508 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947596 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947821 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.947918 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948128 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948215 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948445 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948539 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948757 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.948857 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949065 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949153 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949373 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949461 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949679 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949774 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.949987 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950083 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950293 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950379 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950597 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950728 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950896 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.950993 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951249 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951362 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951526 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951620 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951828 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.951961 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.952123 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.952210 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.952430 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.952516 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.952784 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953026 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953115 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953327 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953423 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953630 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953718 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.953938 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954025 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954235 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954329 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954540 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954635 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954837 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.954960 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955177 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955315 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955485 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955664 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955765 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.955973 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956068 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956286 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956374 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956593 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956683 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956898 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.956991 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957207 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957294 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957511 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957599 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.957902 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958116 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958223 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958423 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958510 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958724 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.958811 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959068 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959259 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959400 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959506 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959703 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.959865 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960010 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960155 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960301 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960420 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960594 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960692 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.960897 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961007 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961191 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961286 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961494 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961581 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961797 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.961884 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962091 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962177 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962391 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962476 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962684 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962770 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.962991 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963077 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963320 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963407 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963712 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.963914 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964009 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964222 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964376 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964509 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964634 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964803 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.964958 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.965262 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.965357 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.965566 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.965754 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.965869 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966074 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966161 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966379 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966465 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966680 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966766 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.966983 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967068 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967313 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967403 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967617 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967703 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.967915 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968001 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968233 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968320 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968539 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968630 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968838 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.968931 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969137 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969223 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969440 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969532 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969736 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.969829 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970034 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970333 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970419 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970637 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970735 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.970939 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971025 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971258 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971345 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971562 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971659 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971859 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.971945 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972158 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972243 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972455 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972542 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972753 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.972838 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973047 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973140 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973350 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973436 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973647 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973731 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.973942 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974028 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974237 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974324 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974547 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974632 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974847 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.974932 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975194 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975298 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975495 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975581 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975794 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.975948 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976130 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976228 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976425 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976511 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976771 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.976865 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977070 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977156 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977374 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977459 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977676 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977774 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.977978 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978065 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978277 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978363 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978580 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978685 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978877 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.978962 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979208 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979296 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979515 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979602 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979820 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.979909 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980118 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980203 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980417 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980504 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980733 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.980817 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981034 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981338 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981437 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981634 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981720 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.981957 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982051 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982275 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982371 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982575 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982662 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982876 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.982971 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983207 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983299 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983510 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983596 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983811 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.983909 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984112 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984199 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984415 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984502 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984718 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.984905 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.985020 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.985165 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.985581 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.985732 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.985875 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986028 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986177 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986274 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986478 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986566 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986798 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.986892 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987139 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987258 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987461 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987548 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987772 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.987882 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988063 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988157 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988366 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988452 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988665 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988752 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.988969 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989054 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989271 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989401 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989602 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989691 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989909 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.989995 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.990215 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.990302 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.990523 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991146 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991375 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991548 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991679 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991778 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.991986 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992073 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992291 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992377 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992589 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992688 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992897 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.992994 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993243 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993346 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993550 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993646 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993876 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.993962 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994173 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994317 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994468 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994563 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994770 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.994856 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995071 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995249 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995393 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995482 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995692 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995781 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.995985 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996121 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996301 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996429 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996597 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996735 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.996911 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997037 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997320 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997408 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997626 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997712 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.997924 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998010 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998236 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998323 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998565 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998650 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.998909 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999003 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999246 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999350 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999564 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999651 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999881 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:21.999975 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000181 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000267 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000482 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000782 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.000880 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001099 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001184 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001400 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001484 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001703 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.001800 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002005 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002098 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002301 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002387 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002612 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002697 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.002926 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.003018 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.003257 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.003385 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.003579 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.003705 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004203 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004322 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004406 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004486 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004605 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004802 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.004899 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005113 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005213 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005435 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005525 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005762 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.005859 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.006069 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.006155 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.006922 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007022 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007248 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007352 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007558 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007645 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007857 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.007943 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008164 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008253 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008469 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008555 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008770 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.008907 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009069 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009157 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009367 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009453 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009681 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009768 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.009981 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.010068 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.010785 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.011751 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.011840 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.011929 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012021 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012104 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012183 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012261 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012342 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012429 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012511 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012590 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012669 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012748 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012830 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.012921 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013010 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013091 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013170 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013249 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013359 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013475 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013552 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013745 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.013864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014049 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014187 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014357 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014511 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014668 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014816 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.014977 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.015164 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.015306 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.015486 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.015635 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.015794 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016021 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016115 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016327 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016413 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016656 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016815 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.016962 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017065 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017265 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017351 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017561 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017648 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017861 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.017948 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018157 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018244 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018459 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018544 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018756 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.018852 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019053 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019156 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019370 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019458 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019687 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019781 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.019998 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020107 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020316 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020492 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020645 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020798 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.020973 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.021098 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.021282 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.021382 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.021615 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.021711 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022016 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022114 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022332 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022430 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022647 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022740 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.022970 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023081 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023402 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023699 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.023916 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024003 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024218 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024319 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024523 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024609 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024825 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.024918 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025131 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025221 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025431 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025518 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025732 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.025820 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026033 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026335 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026420 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026637 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026733 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.026943 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027030 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027274 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027362 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027583 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027682 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.027940 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028131 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028226 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028455 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028565 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028761 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.028869 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029066 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029202 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029371 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029516 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029676 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029835 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.029980 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030158 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030285 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030464 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030589 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030784 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.030892 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031134 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031236 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031427 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031524 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031738 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.031832 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.032048 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.032135 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.032568 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.032677 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.032913 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033007 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033097 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033179 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033296 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033385 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033612 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033706 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.033922 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034008 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034235 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034332 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034540 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034776 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.034864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035116 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035207 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035436 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035523 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035746 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.035833 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036046 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036141 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036352 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036452 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036658 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036748 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.036961 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037049 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037264 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037351 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037595 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037694 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037902 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.037998 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038222 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038539 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038639 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038852 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.038951 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039178 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039317 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039526 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039630 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039840 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.039927 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.040151 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.040355 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.040451 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.040667 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.040794 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041000 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041113 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041323 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041432 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041639 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041728 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.041951 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042038 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042264 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042359 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042570 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042663 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042873 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.042986 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043199 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043296 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043505 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043591 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043806 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.043900 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044103 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044232 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044412 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044499 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044719 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.044853 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045019 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045116 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045327 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045422 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045629 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045726 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.045955 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.046152 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.046265 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.046495 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.046582 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.046806 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047079 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047191 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047416 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047540 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047757 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.047916 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048049 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048138 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048369 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048457 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048669 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048757 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.048969 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049055 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049276 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049407 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049581 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049672 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049882 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.049988 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050182 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050283 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050488 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050589 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050786 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.050939 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051085 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051216 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051408 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051525 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051704 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.051804 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052018 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052135 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052314 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052421 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052612 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052888 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.052991 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.053775 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.053880 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054088 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054195 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054385 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054486 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054688 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054795 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.054989 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055088 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055315 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055412 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055613 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055711 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.055915 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056014 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056225 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056338 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056516 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056622 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056814 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.056914 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057116 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057226 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057458 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057559 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057776 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.057879 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058074 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058181 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058370 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058471 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058677 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058788 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.058983 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059334 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059334 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059650 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059738 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059821 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059911 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.059992 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.060224 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.060313 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.060532 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.060618 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.060844 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.061062 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.061149 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.061392 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.061482 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.061707 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062028 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062241 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062404 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062536 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062617 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.062960 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.063280 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.063458 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.063630 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.063765 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.064265 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.064265 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.064703 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.064858 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.065049 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.065049 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.065453 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.065722 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.065905 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.067756 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.068045 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.068135 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071080 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071232 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071323 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071407 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071489 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071569 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.071658 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.072152 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.072613 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.073116 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.073618 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.074044 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.074236 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.075594 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078486 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078588 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078588 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078715 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078806 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078900 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.078982 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079049 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079185 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079270 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079364 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079446 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079514 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079579 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079644 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079718 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079791 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079859 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079923 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.079988 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080061 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080136 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080204 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080271 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080336 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080471 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080544 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080612 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080677 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080749 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080815 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080889 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.080963 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081030 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081096 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081167 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081234 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081307 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081381 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081454 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081542 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081622 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081701 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081788 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081869 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.081954 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082034 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082119 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082210 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082210 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082355 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082433 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082518 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082604 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082691 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082772 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082851 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.082937 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083030 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083126 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083220 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083380 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083465 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083546 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083630 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083719 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083719 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083852 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083918 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.083983 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084057 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084148 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084236 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084317 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084426 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084496 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084570 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084570 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084706 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084851 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084928 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.084995 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085071 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085139 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085204 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085271 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085345 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085420 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085488 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085635 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085700 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085766 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085839 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085907 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.085972 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086038 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086103 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086168 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086314 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086314 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086446 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086512 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086584 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086584 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086718 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086786 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086851 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086915 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086987 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.086987 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087157 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087228 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087337 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087414 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087480 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087480 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087615 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087682 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087753 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087820 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087885 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.087958 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088025 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088090 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088154 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088218 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088289 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088364 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088430 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088495 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088566 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088632 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088705 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088778 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088844 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088909 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.088981 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089048 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089121 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089220 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089420 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089519 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089741 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.089816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090106 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090204 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090356 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090433 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090666 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090751 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.090980 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091057 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091312 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091403 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091706 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.091939 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092022 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092243 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092328 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092545 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092630 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092853 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.092935 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093156 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093240 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093460 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093545 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093795 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.093885 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094082 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094158 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094402 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094486 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094712 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.094788 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095014 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095126 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095348 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095425 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095739 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.095943 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096035 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096106 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096263 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096343 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096589 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.096787 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097230 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097534 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097693 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.097816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.098005 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.098289 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.098513 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.098690 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.098845 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110501) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:35:22.099032 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.099032 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.099448 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.099739 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.099988 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.100313 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.100622 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.100936 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.101267 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.101634 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.101989 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.102205 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.102534 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.102860 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.103229 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.103543 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.103886 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.104222 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.104534 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.104843 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.105146 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:22.105390 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:35:22.157161 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO dbcounter [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Registered counter for database nova_api May 23 15:35:22.164755 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO dbcounter [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Registered counter for database nova_api May 23 15:35:22.177033 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_db.sqlalchemy.engines [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.184537 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writer thread running {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.185382 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_db.sqlalchemy.engines [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.192128 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writer thread running {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.242271 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.context [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),99fe71d5-840c-46a3-b7af-a004ed27ab79(cell1) {{(pid=110501) load_cells /opt/stack/nova/nova/context.py:464}} May 23 15:35:22.246405 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:35:22.248004 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:35:22.248784 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:35:22.250262 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.context [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),99fe71d5-840c-46a3-b7af-a004ed27ab79(cell1) {{(pid=110502) load_cells /opt/stack/nova/nova/context.py:464}} May 23 15:35:22.250733 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:35:22.252757 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO dbcounter [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Registered counter for database nova_cell0 May 23 15:35:22.254359 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_concurrency.lockutils [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110502) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:35:22.256045 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_concurrency.lockutils [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110502) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:35:22.257313 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO dbcounter [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Registered counter for database nova_cell1 May 23 15:35:22.257575 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_concurrency.lockutils [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110502) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:35:22.259443 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_concurrency.lockutils [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110502) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:35:22.261513 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO dbcounter [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Registered counter for database nova_cell0 May 23 15:35:22.265799 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO dbcounter [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Registered counter for database nova_cell1 May 23 15:35:22.266616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_db.sqlalchemy.engines [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.267466 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_db.sqlalchemy.engines [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.273310 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_db.sqlalchemy.engines [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.274498 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writer thread running {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.274970 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_db.sqlalchemy.engines [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.275789 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writer thread running {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.280117 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writer thread running {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.421253 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writer thread running {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.421957 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO dbcounter [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Registered counter for database nova_cell0 May 23 15:35:22.432232 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO dbcounter [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Registered counter for database nova_cell0 May 23 15:35:22.433592 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_db.sqlalchemy.engines [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110501) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.438571 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writer thread running {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.441250 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG oslo_db.sqlalchemy.engines [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:22.446011 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writer thread running {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:22.876176 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: WARNING keystonemiddleware.auth_token [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:35:22.883444 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: WARNING keystonemiddleware.auth_token [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:35:22.904505 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: WARNING keystonemiddleware.auth_token [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:35:22.912032 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: WARNING keystonemiddleware.auth_token [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:35:23.226872 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: WARNING keystonemiddleware.auth_token [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:35:23.233297 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: WARNING keystonemiddleware.auth_token [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:35:23.239081 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560f85fa09c0 pid: 110501 (default app) May 23 15:35:23.241429 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:35:23.242616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] 10.222.0.32 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001976 May 23 15:35:23.243029 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 1/1] 10.222.0.32 () {50 vars in 788 bytes} [Mon May 23 15:35:23 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:35:23.251680 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: WARNING keystonemiddleware.auth_token [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:35:23.258282 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: WARNING keystonemiddleware.auth_token [None req-4739328a-f999-4639-82ec-5f2959459c67 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:35:23.263593 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560f85fa09c0 pid: 110502 (default app) May 23 15:36:49.735308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:36:49.740699 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001577 May 23 15:36:49.740699 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 2/2] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:36:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:36:49.750047 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_api:SELECT=3 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:49.750883 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_cell1:SELECT=3 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:49.751863 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_cell0:SELECT=3 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:49.752908 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:50.231259 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-884bd9cc-5c03-407d-8b7f-7caba88c83a0 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:36:50.259599 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-884bd9cc-5c03-407d-8b7f-7caba88c83a0 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.514666 May 23 15:36:50.260196 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 1/3] 10.222.0.32 () {60 vars in 1291 bytes} [Mon May 23 15:36:49 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:36:52.919675 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:36:52.920683 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-99dea288-8b3e-431a-8b6e-850189bd7c51 None None] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 May 23 15:36:52.921075 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 3/4] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:36:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:36:52.984589 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-b5337966-0cc4-493b-a2ed-3a974032f35d admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "ram": 256, "swap": 0, "rxtx_factor": 1.0, "vcpus": 1, "os-flavor-access:is_public": true, "name": "cirros256", "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:53.003177 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:36:53.003177 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: warnings.warn(msg) May 23 15:36:53.117189 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-b5337966-0cc4-493b-a2ed-3a974032f35d admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.189876 May 23 15:36:53.118163 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 2/5] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:36:52 2022] POST /compute/v2.1/flavors => generated 407 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:36:53.338316 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-a09aab24-62a3-4395-a712-9169dc263cae admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:53.340779 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:36:53.340779 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:36:53.343492 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_api:SELECT=3 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.344293 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_cell1:SELECT=3 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.345263 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_cell0:SELECT=3 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.345948 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.543000 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-a09aab24-62a3-4395-a712-9169dc263cae admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.416336 May 23 15:36:53.543698 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 4/6] 10.222.0.32 () {64 vars in 1335 bytes} [Mon May 23 15:36:53 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:36:55.991508 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-b5337966-0cc4-493b-a2ed-3a974032f35d admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:36:55.999021 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-b5337966-0cc4-493b-a2ed-3a974032f35d admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001602 May 23 15:36:55.999021 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 3/7] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:36:55 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:36:56.104514 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-e954a333-62f5-49cf-90a4-ed0ebc54e8e7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d1", "os-flavor-access:is_public": true, "ram": 512, "rxtx_factor": 1.0, "vcpus": 1, "name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 5}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:56.122033 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:36:56.122033 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:36:56.231218 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-e954a333-62f5-49cf-90a4-ed0ebc54e8e7 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.232127 May 23 15:36:56.235176 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 5/8] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:36:55 2022] POST /compute/v2.1/flavors => generated 404 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:36:56.249931 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-9c1e1469-3333-4142-8339-9edf8c9a7051 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:56.251602 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:36:56.251602 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: warnings.warn(msg) May 23 15:36:56.355278 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-9c1e1469-3333-4142-8339-9edf8c9a7051 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.110708 May 23 15:36:56.356347 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 4/9] 10.222.0.32 () {64 vars in 1335 bytes} [Mon May 23 15:36:56 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:36:58.663296 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-e954a333-62f5-49cf-90a4-ed0ebc54e8e7 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:36:58.664315 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-e954a333-62f5-49cf-90a4-ed0ebc54e8e7 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001553 May 23 15:36:58.664889 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 6/10] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:36:58 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:36:58.734366 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-97981935-6fa3-46c3-a80d-718bf8b766bc admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 10, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds1G", "ram": 1024, "swap": 0, "rxtx_factor": 1.0, "id": "d2", "vcpus": 1}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:58.820300 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-97981935-6fa3-46c3-a80d-718bf8b766bc admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.148972 May 23 15:36:58.822487 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 5/11] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:36:58 2022] POST /compute/v2.1/flavors => generated 404 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:36:58.835236 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-5edab443-5bd4-4b23-b298-6335c5ae85a6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:36:58.929222 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-5edab443-5bd4-4b23-b298-6335c5ae85a6 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.099786 May 23 15:36:58.930059 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 7/12] 10.222.0.32 () {64 vars in 1335 bytes} [Mon May 23 15:36:58 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:01.391760 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-97981935-6fa3-46c3-a80d-718bf8b766bc admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:01.392678 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-97981935-6fa3-46c3-a80d-718bf8b766bc admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001563 May 23 15:37:01.393100 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 6/13] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:01.479942 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-2765b9a9-2a0e-4853-94ef-ea6c5e9aa3d4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "d3", "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 10, "name": "ds2G", "vcpus": 2, "swap": 0}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:01.554624 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-2765b9a9-2a0e-4853-94ef-ea6c5e9aa3d4 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.152576 May 23 15:37:01.555203 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 8/14] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:01 2022] POST /compute/v2.1/flavors => generated 404 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:01.571468 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-fac6010e-ca6e-4004-8ac3-10b55d8bd70e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:01.679871 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-fac6010e-ca6e-4004-8ac3-10b55d8bd70e admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.112966 May 23 15:37:01.687682 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 7/15] 10.222.0.32 () {64 vars in 1335 bytes} [Mon May 23 15:37:01 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:37:04.163035 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-2765b9a9-2a0e-4853-94ef-ea6c5e9aa3d4 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:04.164901 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-2765b9a9-2a0e-4853-94ef-ea6c5e9aa3d4 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002210 May 23 15:37:04.179724 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 9/16] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:04.255930 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "disk": 20, "id": "d4", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "vcpus": 4}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:04.421107 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.247365 May 23 15:37:04.421704 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 8/17] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:04 2022] POST /compute/v2.1/flavors => generated 404 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:37:04.435041 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-a31908af-5a76-4633-a8e5-210bd88c4409 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:04.526110 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-a31908af-5a76-4633-a8e5-210bd88c4409 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.094954 May 23 15:37:04.532024 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 10/18] 10.222.0.32 () {64 vars in 1335 bytes} [Mon May 23 15:37:04 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:07.083213 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:07.084273 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001523 May 23 15:37:07.084682 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 9/19] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:37:07.163326 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-8c62a3c0-8519-4d51-bb05-90ca1761ee76 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:07.196653 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-8c62a3c0-8519-4d51-bb05-90ca1761ee76 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.105527 May 23 15:37:07.197284 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 11/20] 10.222.0.32 () {60 vars in 1291 bytes} [Mon May 23 15:37:07 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:37:09.903531 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:09.905195 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-d0cdbffd-1dd0-4dff-a5a9-0e2fd27c9901 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002511 May 23 15:37:09.905909 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 10/21] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:09 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:09.997286 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-56f3a2f7-55d0-46d0-b89f-d5129c2af481 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "1", "ram": 512, "vcpus": 1, "name": "m1.tiny"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:10.110296 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-56f3a2f7-55d0-46d0-b89f-d5129c2af481 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.197414 May 23 15:37:10.110951 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 12/22] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:09 2022] POST /compute/v2.1/flavors => generated 402 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:10.126218 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-ab196c09-8a67-4258-a4c6-ba823f3f3647 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:10.262611 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-ab196c09-8a67-4258-a4c6-ba823f3f3647 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.142090 May 23 15:37:10.263583 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 11/23] 10.222.0.32 () {64 vars in 1332 bytes} [Mon May 23 15:37:10 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:12.829806 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-56f3a2f7-55d0-46d0-b89f-d5129c2af481 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:12.831168 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-56f3a2f7-55d0-46d0-b89f-d5129c2af481 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001948 May 23 15:37:12.831695 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 13/24] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:12 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:12.911357 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-b26ceac1-e00d-4caf-a387-ca52b03a25d4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1, "swap": 0, "rxtx_factor": 1.0, "name": "m1.small", "id": "2"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:13.007523 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-b26ceac1-e00d-4caf-a387-ca52b03a25d4 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.167384 May 23 15:37:13.008001 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 12/25] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:12 2022] POST /compute/v2.1/flavors => generated 405 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:13.021130 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-2e1df39d-62ad-4ce8-81cf-3a4f0ef8dbd9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:13.155221 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-2e1df39d-62ad-4ce8-81cf-3a4f0ef8dbd9 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.137614 May 23 15:37:13.156364 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 14/26] 10.222.0.32 () {64 vars in 1332 bytes} [Mon May 23 15:37:13 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:15.479902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-b26ceac1-e00d-4caf-a387-ca52b03a25d4 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:15.481213 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-b26ceac1-e00d-4caf-a387-ca52b03a25d4 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001984 May 23 15:37:15.481689 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 13/27] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:15 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:15.551511 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-12e9b855-0878-4cf0-b7fc-6583560c7bb7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "swap": 0, "ram": 4096, "vcpus": 2, "id": "3", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 40}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:15.651894 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-12e9b855-0878-4cf0-b7fc-6583560c7bb7 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.163792 May 23 15:37:15.652454 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 15/28] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:15 2022] POST /compute/v2.1/flavors => generated 406 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:37:15.668348 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-fd20fa45-8c1c-43c1-bd06-63f8e98e5e18 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:15.763502 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-fd20fa45-8c1c-43c1-bd06-63f8e98e5e18 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.102755 May 23 15:37:15.764030 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 14/29] 10.222.0.32 () {64 vars in 1332 bytes} [Mon May 23 15:37:15 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:18.079261 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-12e9b855-0878-4cf0-b7fc-6583560c7bb7 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:18.080312 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-12e9b855-0878-4cf0-b7fc-6583560c7bb7 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001523 May 23 15:37:18.080720 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 16/30] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:18.143554 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-43cefc86-d3c2-44cf-b9e3-bb5fdce69621 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "name": "m1.large", "disk": 80, "id": "4", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 8192, "swap": 0}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:18.372111 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-43cefc86-d3c2-44cf-b9e3-bb5fdce69621 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.284075 May 23 15:37:18.372561 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 15/31] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:18.385009 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-04b234b6-b3da-4bf2-948c-19440ec4caf6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:18.518682 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-04b234b6-b3da-4bf2-948c-19440ec4caf6 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.137298 May 23 15:37:18.519304 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 17/32] 10.222.0.32 () {64 vars in 1332 bytes} [Mon May 23 15:37:18 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:37:20.886471 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-43cefc86-d3c2-44cf-b9e3-bb5fdce69621 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:20.888342 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-43cefc86-d3c2-44cf-b9e3-bb5fdce69621 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002267 May 23 15:37:20.889260 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 16/33] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:20 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:20.961338 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0, "vcpus": 8, "disk": 160, "ram": 16384, "id": "5"}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:21.088629 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.193083 May 23 15:37:21.089223 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 18/34] 10.222.0.32 () {64 vars in 1282 bytes} [Mon May 23 15:37:20 2022] POST /compute/v2.1/flavors => generated 408 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:21.101758 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-28eb3b42-568d-45a4-bba1-c80d67fb3b8b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:21.105895 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=9,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.289736 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-28eb3b42-568d-45a4-bba1-c80d67fb3b8b admin admin] 10.222.0.32 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.192196 May 23 15:37:21.290895 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 17/35] 10.222.0.32 () {64 vars in 1332 bytes} [Mon May 23 15:37:21 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:37:45.836389 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:45.837412 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 May 23 15:37:45.837818 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 19/36] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:37:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:45.901726 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-cbba2834-17df-49be-ad78-f666e1c9cc82 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:45.919879 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-cbba2834-17df-49be-ad78-f666e1c9cc82 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.075108 May 23 15:37:45.920527 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 18/37] 10.222.0.32 () {60 vars in 1291 bytes} [Mon May 23 15:37:45 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:37:46.924763 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:46.925762 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-e1e63bb2-cd73-4ac2-a2f5-47613365e6ba admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 May 23 15:37:46.926163 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 20/38] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:47.137944 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-31849785-0930-41b6-8586-708a9fa69864 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:47.143504 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.wsgi [None req-31849785-0930-41b6-8586-708a9fa69864 admin admin] HTTP exception thrown: Flavor 42 could not be found. May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-31849785-0930-41b6-8586-708a9fa69864 admin admin] Request method failure captured: May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept: application/json May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept-Encoding: gzip, deflate May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Connection: keep-alive May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Content-Length: 0 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Host: 10.222.0.32 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-Api-Version: compute 2.90 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-System-Scope: None May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: User-Agent: python-novaclient May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Auth-Token: gAAAAABii6pKUhuj7l80_3eIBIh4yFw1-vabU6bD-PFrfBGafOEFug2nkvDdK3zE40rInGw0NDYc66khDjF-s2A18Lc3HpVSUaEN9SKoL1_IVZedMX8PaBT4Ngnlda1TJOq5gtCsMMNFbG6VNO0PoYMuy_TZ7_eJ0rn2nGBW438dbI__o_JRcY4 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Id: None May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Name: None May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Identity-Status: Confirmed May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Is-Admin-Project: True May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Id: default May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Name: Default May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Name: admin May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Role: member,admin,reader May 23 15:37:47.146816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Roles: member,admin,reader May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant: admin May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Name: admin May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User: admin May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Id: default May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Name: Default May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Name: admin May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: method: > May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: exception: Flavor 42 could not be found. May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: action_args: {'id': '42'} May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:47.148441 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:47.149528 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:47.150902 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-31849785-0930-41b6-8586-708a9fa69864 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=110502) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:47.152179 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-31849785-0930-41b6-8586-708a9fa69864 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.063360 May 23 15:37:47.152179 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 19/39] 10.222.0.32 () {60 vars in 1190 bytes} [Mon May 23 15:37:47 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 15:37:47.158506 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-11691ed4-4e37-44be-b53b-b250669fc9c5 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:47.164052 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.181427 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-11691ed4-4e37-44be-b53b-b250669fc9c5 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026706 May 23 15:37:47.181992 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 21/40] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:37:47.192868 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-a2d6f727-9395-4192-9024-b1ab81557ae2 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:47.214097 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-a2d6f727-9395-4192-9024-b1ab81557ae2 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025881 May 23 15:37:47.214660 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 20/41] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:37:47.224573 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-b7dc6245-110c-42c9-8699-ab7c35f3f82c admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:47.233017 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.wsgi [None req-b7dc6245-110c-42c9-8699-ab7c35f3f82c admin admin] HTTP exception thrown: Flavor 42 could not be found. May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-b7dc6245-110c-42c9-8699-ab7c35f3f82c admin admin] Request method failure captured: May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept: application/json May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept-Encoding: gzip, deflate May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Connection: keep-alive May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Content-Length: 0 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Host: 10.222.0.32 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-Api-Version: compute 2.90 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-System-Scope: None May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: User-Agent: python-novaclient May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Auth-Token: gAAAAABii6pKUhuj7l80_3eIBIh4yFw1-vabU6bD-PFrfBGafOEFug2nkvDdK3zE40rInGw0NDYc66khDjF-s2A18Lc3HpVSUaEN9SKoL1_IVZedMX8PaBT4Ngnlda1TJOq5gtCsMMNFbG6VNO0PoYMuy_TZ7_eJ0rn2nGBW438dbI__o_JRcY4 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Id: None May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Name: None May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Identity-Status: Confirmed May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Is-Admin-Project: True May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Id: default May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Name: Default May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Name: admin May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Role: member,admin,reader May 23 15:37:47.236878 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Roles: member,admin,reader May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant: admin May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Name: admin May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User: admin May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Id: default May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Name: Default May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Name: admin May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: method: > May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: exception: Flavor 42 could not be found. May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: action_args: {'id': '42'} May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:47.238946 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:47.239864 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:47.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-b7dc6245-110c-42c9-8699-ab7c35f3f82c admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=110501) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:47.242645 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-b7dc6245-110c-42c9-8699-ab7c35f3f82c admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.018349 May 23 15:37:47.242645 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 22/42] 10.222.0.32 () {60 vars in 1190 bytes} [Mon May 23 15:37:47 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 15:37:48.119737 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-a2d6f727-9395-4192-9024-b1ab81557ae2 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:48.120753 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-a2d6f727-9395-4192-9024-b1ab81557ae2 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 May 23 15:37:48.121167 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 21/43] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:48.312932 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-4b60e214-3abb-4543-b844-5a496c8b95ba admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:48.319541 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.wsgi [None req-4b60e214-3abb-4543-b844-5a496c8b95ba admin admin] HTTP exception thrown: Flavor 84 could not be found. May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-4b60e214-3abb-4543-b844-5a496c8b95ba admin admin] Request method failure captured: May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept: application/json May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept-Encoding: gzip, deflate May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Connection: keep-alive May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Content-Length: 0 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Host: 10.222.0.32 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-Api-Version: compute 2.90 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-System-Scope: None May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: User-Agent: python-novaclient May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Auth-Token: gAAAAABii6pMFtm_RyMRGpAR2OtI4h2piryZBd-MSiVEtZbOSe0mhItwIc4whvsE5l4-JdzSpZmZovgslroP12kw67CtH_VVdhdpMZGlYrw-mRKcOJq3FY7Rl2lhr_zvW19arV-hY-C8xdzuzxWgc5W5u0QKQkmDLIrXlP8JLOjYAVs1xdnrAY0 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Id: None May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Name: None May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Identity-Status: Confirmed May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Is-Admin-Project: True May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Id: default May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Name: Default May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Name: admin May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Role: member,admin,reader May 23 15:37:48.321090 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Roles: member,admin,reader May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant: admin May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Name: admin May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User: admin May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Id: default May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Name: Default May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Name: admin May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: method: > May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: exception: Flavor 84 could not be found. May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: action_args: {'id': '84'} May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:48.322883 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:48.324849 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:48.325895 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-4b60e214-3abb-4543-b844-5a496c8b95ba admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=110501) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:48.327284 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-4b60e214-3abb-4543-b844-5a496c8b95ba admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.061893 May 23 15:37:48.327284 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 23/44] 10.222.0.32 () {60 vars in 1190 bytes} [Mon May 23 15:37:48 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) May 23 15:37:48.333174 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-db712d3c-5aeb-4ad4-a220-4c26e891649f admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:48.348382 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-db712d3c-5aeb-4ad4-a220-4c26e891649f admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019506 May 23 15:37:48.348969 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 22/45] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:48 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:37:48.358786 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-35bbdb72-b822-4429-af2f-228b55b2be54 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:48.373544 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-35bbdb72-b822-4429-af2f-228b55b2be54 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018658 May 23 15:37:48.374076 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 24/46] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:48 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:37:48.384164 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-ba1bb2aa-e80f-44d6-8c40-447fdeedfdea admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:48.389581 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.wsgi [None req-ba1bb2aa-e80f-44d6-8c40-447fdeedfdea admin admin] HTTP exception thrown: Flavor 84 could not be found. May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-ba1bb2aa-e80f-44d6-8c40-447fdeedfdea admin admin] Request method failure captured: May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept: application/json May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept-Encoding: gzip, deflate May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Connection: keep-alive May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Content-Length: 0 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Host: 10.222.0.32 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-Api-Version: compute 2.90 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-System-Scope: None May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: User-Agent: python-novaclient May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Auth-Token: gAAAAABii6pMFtm_RyMRGpAR2OtI4h2piryZBd-MSiVEtZbOSe0mhItwIc4whvsE5l4-JdzSpZmZovgslroP12kw67CtH_VVdhdpMZGlYrw-mRKcOJq3FY7Rl2lhr_zvW19arV-hY-C8xdzuzxWgc5W5u0QKQkmDLIrXlP8JLOjYAVs1xdnrAY0 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Id: None May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Name: None May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Identity-Status: Confirmed May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Is-Admin-Project: True May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Id: default May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Name: Default May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Name: admin May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Role: member,admin,reader May 23 15:37:48.391176 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Roles: member,admin,reader May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant: admin May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Name: admin May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User: admin May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Id: default May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Name: Default May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Name: admin May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: method: > May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: exception: Flavor 84 could not be found. May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: action_args: {'id': '84'} May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:48.392814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:48.393605 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:48.395309 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-ba1bb2aa-e80f-44d6-8c40-447fdeedfdea admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=110502) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:48.396394 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-ba1bb2aa-e80f-44d6-8c40-447fdeedfdea admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012022 May 23 15:37:48.396394 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 23/47] 10.222.0.32 () {60 vars in 1190 bytes} [Mon May 23 15:37:48 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) May 23 15:37:49.270271 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-35bbdb72-b822-4429-af2f-228b55b2be54 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:49.271421 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-35bbdb72-b822-4429-af2f-228b55b2be54 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001530 May 23 15:37:49.271841 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 25/48] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:49.464889 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-c30531c3-8e1b-493e-8de0-dace937b3a20 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:49.473689 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.wsgi [None req-c30531c3-8e1b-493e-8de0-dace937b3a20 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-c30531c3-8e1b-493e-8de0-dace937b3a20 admin admin] Request method failure captured: May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept: application/json May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Accept-Encoding: gzip, deflate May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Connection: keep-alive May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Content-Length: 0 May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Host: 10.222.0.32 May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-Api-Version: compute 2.90 May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Openstack-System-Scope: None May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: User-Agent: python-novaclient May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Auth-Token: gAAAAABii6pNGoms8E9RuPmDcfKESYCfBF4GNDLSUFsm--1rpdIHWXKx-coaz7NQo1upwZMblpMPnZ9eTyTEUCJacnPjgK6EKCcGD2AlsybdBPpkIuJ3fsjCFW-14M7lwiuu5pEKsEmnokhLP4VTgFS8GGHpqA-22kRDfFapGxvqloZcCBzb3jU May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Id: None May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Domain-Name: None May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Identity-Status: Confirmed May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Is-Admin-Project: True May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Id: default May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Domain-Name: Default May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Project-Name: admin May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Role: member,admin,reader May 23 15:37:49.475814 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Roles: member,admin,reader May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant: admin May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-Tenant-Name: admin May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User: admin May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Id: default May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Domain-Name: Default May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: X-User-Name: admin May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: method: > May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: exception: Flavor 451 could not be found. May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: action_args: {'id': '451'} May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:49.477301 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:49.478123 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: ERROR nova.api.openstack.wsgi May 23 15:37:49.479192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-c30531c3-8e1b-493e-8de0-dace937b3a20 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=110502) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:49.480420 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-c30531c3-8e1b-493e-8de0-dace937b3a20 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.067487 May 23 15:37:49.480420 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 24/49] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:37:49 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 15:37:49.488670 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-bc3e186a-6712-4b1a-9684-9af40d91f5f1 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:49.504149 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-bc3e186a-6712-4b1a-9684-9af40d91f5f1 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019749 May 23 15:37:49.504704 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 26/50] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:49 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:37:49.515192 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:49.529698 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018525 May 23 15:37:49.530236 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 25/51] 10.222.0.32 () {60 vars in 1210 bytes} [Mon May 23 15:37:49 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:37:49.543680 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-696bf866-5961-440b-87e6-84f6e5241ff0 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:49.552401 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.wsgi [None req-696bf866-5961-440b-87e6-84f6e5241ff0 admin admin] HTTP exception thrown: Flavor 451 could not be found. May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-696bf866-5961-440b-87e6-84f6e5241ff0 admin admin] Request method failure captured: May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept: application/json May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Accept-Encoding: gzip, deflate May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Connection: keep-alive May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Content-Length: 0 May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Host: 10.222.0.32 May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-Api-Version: compute 2.90 May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Openstack-System-Scope: None May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: User-Agent: python-novaclient May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Auth-Token: gAAAAABii6pNGoms8E9RuPmDcfKESYCfBF4GNDLSUFsm--1rpdIHWXKx-coaz7NQo1upwZMblpMPnZ9eTyTEUCJacnPjgK6EKCcGD2AlsybdBPpkIuJ3fsjCFW-14M7lwiuu5pEKsEmnokhLP4VTgFS8GGHpqA-22kRDfFapGxvqloZcCBzb3jU May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Id: None May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Domain-Name: None May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Identity-Status: Confirmed May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Is-Admin-Project: True May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Openstack-Nova-Api-Version: 2.90 May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Id: default May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Domain-Name: Default May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Project-Name: admin May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Role: member,admin,reader May 23 15:37:49.554567 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Roles: member,admin,reader May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/75147bc201bf4c1086a8eadbdc968fcf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_75147bc201bf4c1086a8eadbdc968fcf", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant: admin May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Id: 75147bc201bf4c1086a8eadbdc968fcf May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-Tenant-Name: admin May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User: admin May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Id: default May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Domain-Name: Default May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Id: f9d0e31020494ca584e4bf621abe8d70 May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: X-User-Name: admin May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: method: > May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: exception: Flavor 451 could not be found. May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: action_args: {'id': '451'} May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:37:49.556253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:49.557308 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: ERROR nova.api.openstack.wsgi May 23 15:37:49.558679 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-696bf866-5961-440b-87e6-84f6e5241ff0 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=110501) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:37:49.559921 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-696bf866-5961-440b-87e6-84f6e5241ff0 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.018399 May 23 15:37:49.559921 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 27/52] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:37:49 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 23 15:37:50.443214 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:50.444270 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 May 23 15:37:50.444663 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 26/53] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:50 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:50.636505 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-8413e63e-d23e-4b70-a7d5-2ba4c7a6c363 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:50.776297 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-8413e63e-d23e-4b70-a7d5-2ba4c7a6c363 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.194008 May 23 15:37:50.776854 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 28/54] 10.222.0.32 () {64 vars in 1235 bytes} [Mon May 23 15:37:50 2022] POST /compute/v2.1/flavors => generated 404 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:51.672807 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:51.673809 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 May 23 15:37:51.674200 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 27/55] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:51.865009 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-1f5335db-d29e-47c3-8c74-87346e546cf8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:51.977896 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-1f5335db-d29e-47c3-8c74-87346e546cf8 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.166895 May 23 15:37:51.978370 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 29/56] 10.222.0.32 () {64 vars in 1235 bytes} [Mon May 23 15:37:51 2022] POST /compute/v2.1/flavors => generated 405 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:52.862811 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:52.863852 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001741 May 23 15:37:52.864273 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 28/57] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:53.071012 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-c59097e2-a08b-4bb6-853a-85c149c601d1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:53.164033 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-c59097e2-a08b-4bb6-853a-85c149c601d1 admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.160841 May 23 15:37:53.164620 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 30/58] 10.222.0.32 () {64 vars in 1235 bytes} [Mon May 23 15:37:53 2022] POST /compute/v2.1/flavors => generated 407 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:54.054967 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:54.055694 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001753 May 23 15:37:54.056084 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 29/59] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:54 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:54.248468 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-3fcbe777-da04-4f76-8205-c77acafec0dd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:37:54.366700 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-3fcbe777-da04-4f76-8205-c77acafec0dd admin admin] 10.222.0.32 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.172810 May 23 15:37:54.367201 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 31/60] 10.222.0.32 () {64 vars in 1235 bytes} [Mon May 23 15:37:54 2022] POST /compute/v2.1/flavors => generated 409 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:37:55.252842 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:55.254412 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002061 May 23 15:37:55.254877 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 30/61] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:37:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:37:55.444259 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-3ab22c9d-bb9c-4aa3-808f-78c1fd7bbb95 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:37:55.463283 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-3ab22c9d-bb9c-4aa3-808f-78c1fd7bbb95 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.071314 May 23 15:37:55.463736 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 32/62] 10.222.0.32 () {60 vars in 1202 bytes} [Mon May 23 15:37:55 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:38:59.493075 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.494007 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-44ebca38-b29a-4792-862b-4e85405bfa29 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001445 May 23 15:38:59.494403 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 31/63] 10.222.0.32 () {54 vars in 930 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:38:59.502731 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:59.557070 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-f8c57bb7-7845-4418-a4cc-178d2994c81c admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.577423 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-f8c57bb7-7845-4418-a4cc-178d2994c81c admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.076548 May 23 15:38:59.578068 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 33/64] 10.222.0.32 () {60 vars in 1291 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:38:59.600269 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG dbcounter [-] [110502] Writing DB stats nova_api:SELECT=19,nova_api:SAVEPOINT=1,nova_api:INSERT=1,nova_api:RELEASE=1 {{(pid=110502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:59.604041 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-2e724597-2c98-46fa-91d4-fb4a9e81d714 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.612228 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-2e724597-2c98-46fa-91d4-fb4a9e81d714 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013427 May 23 15:38:59.612771 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 32/65] 10.222.0.32 () {60 vars in 1282 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:38:59.625119 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-c5825f2c-f0f4-4e13-b669-3b0a9914c083 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.633593 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-c5825f2c-f0f4-4e13-b669-3b0a9914c083 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012413 May 23 15:38:59.641210 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 34/66] 10.222.0.32 () {60 vars in 1285 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:38:59.646208 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.655730 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013216 May 23 15:38:59.656446 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 33/67] 10.222.0.32 () {60 vars in 1285 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:38:59.670021 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-378d5687-a920-49e2-b8ab-9f62572b234e admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:38:59.678594 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-378d5687-a920-49e2-b8ab-9f62572b234e admin admin] 10.222.0.32 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012204 May 23 15:38:59.681069 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 35/68] 10.222.0.32 () {60 vars in 1282 bytes} [Mon May 23 15:38:59 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:42:48.679816 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:42:48.681356 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002009 May 23 15:42:48.681773 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 34/69] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:42:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:42:48.841658 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_api:SELECT=6 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:42:48.965229 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:42:48.966390 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:42:48.966390 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:42:48.967616 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:42:48.968050 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:42:48.968845 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:42:48.969253 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:42:49.005124 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-018feb3b-f9af-4e69-af79-a8328c4e5ea8 admin admin] 10.222.0.32 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.164886 May 23 15:42:49.005702 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 36/70] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:42:48 2022] GET /compute/v2.1/os-services => generated 794 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:42:49.883373 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:42:49.884365 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 May 23 15:42:49.884757 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 35/71] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:42:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:42:50.083608 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:42:50.084798 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:42:50.084798 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:42:50.085973 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:42:50.086912 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:42:50.087707 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:42:50.088128 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:42:50.109234 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-bf9e7f1e-fa3f-4234-b050-29a1d8e57d59 admin admin] 10.222.0.32 "GET /compute/v2.1/os-services" status: 200 len: 794 microversion: 2.90 time: 0.081164 May 23 15:42:50.109973 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 37/72] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:42:50 2022] GET /compute/v2.1/os-services => generated 794 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:43:01.418497 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:43:01.419533 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001426 May 23 15:43:01.419925 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 36/73] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:43:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:43:01.563841 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_cell0:SELECT=4 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:01.564560 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_cell1:SELECT=4 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:01.565177 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG dbcounter [-] [110501] Writing DB stats nova_api:SELECT=8 {{(pid=110501) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:43:01.642473 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:43:01.643028 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:43:01.643147 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-83ab2189-d11d-47a5-a1c7-8eb9c990ab71 admin admin] 10.222.0.32 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.90 time: 0.083647 May 23 15:43:01.643625 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 38/74] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:43:01 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:43:02.522740 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: DEBUG nova.api.openstack.wsgi [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] Calling method '>' {{(pid=110502) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:43:02.523776 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: INFO nova.api.openstack.requestlog [None req-f2e7f352-988f-464d-a783-a04fe6dca704 admin admin] 10.222.0.32 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001449 May 23 15:43:02.524317 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: [pid: 110502|app: 0|req: 37/75] 10.222.0.32 () {56 vars in 1074 bytes} [Mon May 23 15:43:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:43:02.712794 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG nova.api.openstack.wsgi [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] Calling method '>' {{(pid=110501) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:43:02.713949 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:43:02.713949 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: warnings.warn(msg) May 23 15:43:02.715188 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:43:02.715629 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:43:02.716419 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:43:02.716839 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: DEBUG oslo_concurrency.lockutils [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110501) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:43:02.738779 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: INFO nova.api.openstack.requestlog [None req-b3bf9579-fee4-45e3-8c11-20f7f5da8cce admin admin] 10.222.0.32 "GET /compute/v2.1/os-services" status: 200 len: 1014 microversion: 2.90 time: 0.077615 May 23 15:43:02.739286 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: [pid: 110501|app: 0|req: 39/76] 10.222.0.32 () {60 vars in 1193 bytes} [Mon May 23 15:43:02 2022] GET /compute/v2.1/os-services => generated 1014 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:44:01.697549 dv-cinder-835881-14-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 23 15:44:01.698145 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: Mon May 23 15:44:01 2022 - graceful shutdown triggered... May 23 15:44:01.698598 dv-cinder-835881-14-iscsi devstack@n-api.service[110501]: Gracefully killing worker 1 (pid: 110501)... May 23 15:44:01.699071 dv-cinder-835881-14-iscsi devstack@n-api.service[110502]: Gracefully killing worker 2 (pid: 110502)... May 23 15:44:02.699188 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: worker 1 buried after 1 seconds May 23 15:44:02.699188 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: worker 2 buried after 1 seconds May 23 15:44:02.699188 dv-cinder-835881-14-iscsi devstack@n-api.service[110500]: goodbye to uWSGI. May 23 15:44:02.700757 dv-cinder-835881-14-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 23 15:44:02.701556 dv-cinder-835881-14-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 23 15:44:02.704299 dv-cinder-835881-14-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 23 15:44:02.720618 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 15:44:02 2022] *** May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: nodename: dv-cinder-835881-14-iscsi May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: machine: x86_64 May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: clock source: unix May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: pcre jit disabled May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: detected number of CPU cores: 4 May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: current working directory: / May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: detected binary path: /usr/bin/uwsgi-core May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: your processes number limit is 47680 May 23 15:44:02.737256 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: your memory page size is 4096 bytes May 23 15:44:02.738192 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: detected max file descriptor number: 2048 May 23 15:44:02.738192 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: lock engine: pthread robust mutexes May 23 15:44:02.738192 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: thunder lock: enabled May 23 15:44:02.738192 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 23 15:44:02.738192 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 15:44:02.787069 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: Python main interpreter initialized at 0x55f7591f99c0 May 23 15:44:02.787069 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: python threads support enabled May 23 15:44:02.787069 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: your server socket listen backlog is limited to 100 connections May 23 15:44:02.787069 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: your mercy for graceful operations on workers is 90 seconds May 23 15:44:02.787642 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: mapped 403077 bytes (393 KB) for 2 cores May 23 15:44:02.787642 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: *** Operational MODE: preforking *** May 23 15:44:02.787805 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: *** uWSGI is running in multiple interpreter mode *** May 23 15:44:02.787805 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: spawned uWSGI master process (pid: 124944) May 23 15:44:02.788022 dv-cinder-835881-14-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 23 15:44:02.788628 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: spawned uWSGI worker 1 (pid: 124966, cores: 1) May 23 15:44:02.789172 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: spawned uWSGI worker 2 (pid: 124967, cores: 1) May 23 15:44:02.790187 dv-cinder-835881-14-iscsi devstack@n-api.service[124944]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 15:44:03.982770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 15:44:04.007226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 23 15:44:04.780498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:44:04.780498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: from cryptography.utils import int_from_bytes May 23 15:44:04.780885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:44:04.780885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: from cryptography.utils import int_from_bytes May 23 15:44:04.835978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:44:04.835978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: from cryptography.utils import int_from_bytes May 23 15:44:04.836413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:44:04.836413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: from cryptography.utils import int_from_bytes May 23 15:44:07.202429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:44:07.203219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:44:07.203336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:44:07.203615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:44:07.203877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:44:07.204270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.204600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.205139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.205309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.205661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.205993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.206320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.206638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.206969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.207255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.207573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.207904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835881-14-iscsi {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.208229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.208540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.208872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.209484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.209484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.209791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.210099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.210603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.211043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.211385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.211774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.212046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.212362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.212673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.212993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.213301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.213627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.213938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.214348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835881-14-iscsi {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.214672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.214975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.215316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.215744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.216059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.216369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.216690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.217014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.217326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.217657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.217974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.218293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.218637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.218917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.219250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.219562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.219882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.220189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.220493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.220739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.221049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.221372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.221728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.222053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.222306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.222557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.222872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.223216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.223529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.223943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.224139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.224443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.225002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.225553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.226157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.226634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.226963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.227315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.227648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.227960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.228283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.228850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.32 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.229254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.32 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.229721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.230282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.230756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.231256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.231773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.232247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.232705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.233168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.233617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.234087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.234552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.234999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.235425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.235895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.236341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.236915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.237997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.238500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.238993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.239506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.240004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.240496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.240976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.241459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.241955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.242708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.244135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.244650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.245156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.245696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.246222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.246549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.246869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.247222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.247545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.247879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.248197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.248510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.248838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.249153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.249465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.249792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.250107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.250418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.250742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.251049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.251385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.251711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.252028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.252337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.252643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.253563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.253880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.254193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.254504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.254814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.255150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.255465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.255790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.256108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.256417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.256725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.257047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.257391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.257728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.258079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.258402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.258741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.259079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.259441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.259756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.260073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.260409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.260762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.261126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.261448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.261817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.262144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.262482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.262806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.263151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.263481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.263797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.264120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.264452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.264780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.265098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.265444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.265824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.266111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.266431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.266759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.267071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.267421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.267792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.268058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.268439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.268732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.269058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.269427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.269754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.270070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.270383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.270699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.271037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.271381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.271705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.272028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.272348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.272661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.272975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.273330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.273626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.273935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.274243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.274604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.274866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.275214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.275547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.275861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.276199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.276513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.276827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.277139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.277525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.277761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.278067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.278375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.278689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.279016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.279355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.279666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.279993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.280303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.280613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.280932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.281246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.281561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.281881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.282194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.282520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.282837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.283174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.283518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.283856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.284695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.285027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.285359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.285669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.285994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.286288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.286594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.286910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.287228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.287544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.287854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.288164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.288461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.288770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.289075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.289397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.289709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.290018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.290327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.290632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.290978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.291310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.291656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.291997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.292848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.292848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.293152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.293307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.293561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.293925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.294237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.294564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.294895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.295235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.295564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.295880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.296211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.296526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.296852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.297159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.297478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.297827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.298282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.298454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.298851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.299109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.299432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.300298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.300620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.300933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.301248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.301569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.301883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.302204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.302524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.302853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.303534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.303534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.303998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.304472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.304472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.304792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.305038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.305353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.305678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.305993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.306310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.306622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.306934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.307267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.307579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.307940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.308206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.308509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.308813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.309129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.309438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.309744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.310051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.310363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.310686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.311064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.311433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.311753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.312061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.312375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.312700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.313012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.313319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.313625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.313970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.314336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.314588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.314897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.315237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.315588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.315902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.316222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.316537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.316847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.317172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.317489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.317799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.318116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.318575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.318733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.319613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.319613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.319944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.320098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.320299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.320607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.320921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.321241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.321828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.322165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.322491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.322811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.323170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.323477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.323810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.324142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.324505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.324790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.325098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.325408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.325714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.326025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.326332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.326693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.326938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.327276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.327583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.327888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.328200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.328506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.328830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.329114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.329427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.329734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.330063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.330375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.330683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.331044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.331379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.331727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.332036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.332390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.332723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.333067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.333376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.333682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.334008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.334318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.334677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.334923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.335305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.335666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.335976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.336292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.336604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.336921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.337232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.337527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.337825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.338130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.338450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.338939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.339136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.339456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.339769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.340065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.340377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.340687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.340992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.341314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.341624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.341932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.342243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.342555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.342864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.343196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.343535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.343850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.344158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.344468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.344871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.345078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.345571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.345782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.346111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.346436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.347343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.347632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.347939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.348250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.348720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.348916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.349282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.349455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.349765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.350071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.350379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.350714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.351026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.351364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.351673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.351979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.352329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.352640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.353016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.353327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.353646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.353968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.354280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.354628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.355626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.355626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.355626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.356069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.356219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.356425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.356729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.357059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.357442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.357785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.358085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.358434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.358965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.359178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.359539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.359759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.360072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.360386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.360696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.361011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.361326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.361641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.365244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.366668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.368781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.369287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.369287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.369808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.370059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.370269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.370579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.370885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.371218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.371531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.371858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.372164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.372468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.372772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.373098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.373405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.373726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.374619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.374619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.374930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.375399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.375634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.376068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.376265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.376590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.376916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.377223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.378978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.379452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.379880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.380040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.380339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.380668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.381324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.381324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.382496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.382496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.382496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.382953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.383146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.383380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.384147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.384147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.384432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.384628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.384946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.385259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.385572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.385899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.386262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.386523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.386840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.387181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.387493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.387801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.388132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.388428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.388786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.389064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.389540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.389701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.390089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.390333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.390645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.390954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.391293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.391604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.391953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.392261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.392585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.392925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.393261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.393640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.393962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.394295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.394610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.394932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.395317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.395706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.395934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.396249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.396560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.396854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.397160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.397470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.397778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.398082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.398399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.398662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:44:07.399316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.399510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:44:07.399871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.400055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:44:07.400367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.400367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.400666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:44:07.400666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:44:07.400973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.400973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.401251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.401453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.401612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.401764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.401915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.402071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.402218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.402360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.402512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.402711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.402865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.403019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.403336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.403505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.403650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.403801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.403945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.404172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.404333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.404487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.404639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.404781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.404933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-835881-14-iscsi {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.405087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.405229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.405380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.405530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.405672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.405958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.406116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.406267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.406423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.406729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.406878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.407030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.407229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.407400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.407543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.407684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.408014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.408190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.408190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.408668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.408929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.409758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.410117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.410291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.410564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.410722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.410875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.411018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.411194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.411341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.411494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.411645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.411796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.411938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.412080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.412221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.412373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-835881-14-iscsi {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.412526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.412526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.412807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.412807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.413089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.413240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.413511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.413511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.413797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.413954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.414098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.414239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.414398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.414540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.414689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.414840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.414984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.415165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.415322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.415466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.415618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.415775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.416053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.416203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.416364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.416508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.416891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.417044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.417194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.417337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.417487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.417637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.417802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.417958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.418109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.418251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.418404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.418711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.418864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.418864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.419293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.419449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.419600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.419900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.420275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.420425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.420592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.420852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.421023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.421178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.421323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.421475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.421620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.421763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.421921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.422065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.422208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.422364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.422507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.422659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.422659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.422967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.423190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.423340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.423610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.423773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.423932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.424082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.424225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.424375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.424517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.424733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.425410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.425569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.425721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.433234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.434376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.434583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.434749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.434900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.435064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.435252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.435402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.32 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.435554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.435725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.32 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.435880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.436035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.436179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.436321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.436465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.436617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.436776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.436922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.437064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.437064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.437343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.437343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.437343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.437773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.437925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.438076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.438227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.438227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.438521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.438665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.438665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.438935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.439088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.439299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.439488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.439635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.439846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.439995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.440169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.440169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.440504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.440657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.440808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.440978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.441152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.441343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.441492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.441650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.441794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.441937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.442078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.442236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.442380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.442523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.443931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.444119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.444277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.444423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.444568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.444711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.444880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.445025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.445176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.445320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.445462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.445604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.445755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.445908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.446059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.446203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.446602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.446757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.446902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.447057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.447241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.447388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.447529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.447680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.448213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.448403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.448573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.448780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.448942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.449088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.449230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.449372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.449560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.449723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.449723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.450020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.450165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.450314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.450459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.450601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.450753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.450903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.451052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.451052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.451416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.451561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.451758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.451911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.452054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.452196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.452345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.452488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.452488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.452767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.452926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.453732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.453898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.454070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.454224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.454368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.454511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.454651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.454793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.454948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.455107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.455258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.455411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.455411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.455682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.455836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.456047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.456193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.456337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.456485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:44:07.456628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.456780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.456925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.457980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.458729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.459184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.460910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.461949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.462980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.463145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.463783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.463937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.464976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.465871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.466900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.467976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.468900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.469909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.470919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.471063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.471339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.472983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.473866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.474386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.474531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.474673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.474815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.474992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.475921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.476960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.477842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.478884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.479840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.480989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.481865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.482927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.483879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.484056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.484498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.484656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.484805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.484973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.485915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.486992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.487238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.487895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.488995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.489145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.489293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.489467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.489638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.489788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.490940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.491259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.491259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.491554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.491718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.491867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.492022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.492169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.492565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.492722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.492880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.493966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.494135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.494618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.494775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.494935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.495949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.496872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124966) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.497348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.498477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.499632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.500760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.501894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.503144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.504292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.505555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.506672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.507684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.507684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.507684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.508118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.508332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.508700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.509027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.509364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.509698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.510057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.510369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.510684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.510999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.511339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.511656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.511969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.512280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.512594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.512910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.513229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.513540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.513851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.514164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.514480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.514794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.515122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.515441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.515774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.516093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.516404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.516714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.517023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.517370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.517685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.518027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.518362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.520034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.521005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.521348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.522123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.522465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.522928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.523310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.524585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.525032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.525267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.525607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.525945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.526284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.526616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.526936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.527291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.527616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.527944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.528268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.528587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.528902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.529225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.529546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.529862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.530186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.530498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.530824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.531164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.531484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.531800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.532122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.532442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.532755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.533089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.533409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.533723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.534546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.534870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.535349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.535526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.535819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.536151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.536479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.536797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.537205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.537522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.537837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.538151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.538466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.538773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.539082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.539418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.539738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.540090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.540436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.540753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.541065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.541378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.541690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.542042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.542357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.542723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.543038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.543389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.543717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.544036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.544145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO dbcounter [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Registered counter for database nova_api May 23 15:44:07.544355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.544674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.544986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.545313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.545629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.545950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.546263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.546656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.547006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.547355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.547677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.548009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.548343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.548677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.548991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.549308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.549643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.550619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.550942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.551288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.551612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.551933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.552256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.552576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.553006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.553596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.554181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.554773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.555461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.555832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.556161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.556483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.556806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.557127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.557451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.557769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.558220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.558542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.558861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.559201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.559529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.560800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.561183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.561791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.562377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.563230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.563582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.563913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.564242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.564567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.564887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.565191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_db.sqlalchemy.engines [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124966) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.565300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.565591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.565911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.566225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.566540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.566874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.567223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.567558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.567991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.568326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.568627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.569056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.569802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.569802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.570114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.570523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.570779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.571114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.573358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writer thread running {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.580303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.580664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.581047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.581398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.581818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.582339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.582780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.583271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.583735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.584211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.584653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.585213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.585653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.586110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.586555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.586991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.587471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.587934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.588363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.588785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.589212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.589644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.590061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.590483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.590903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.591361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.591793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.592212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.592647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.593067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.593504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.593935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.594432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.594871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.595328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.595770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.596188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.597131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.597549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.598025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.598484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.598899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.599361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.599813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.600254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.600679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.601120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.601567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.601993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.602427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.602864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.603320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.603743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.604164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.604590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.605012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.605427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.605844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.606264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.606711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.607164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.607598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.608115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.608551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.608970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.609381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.609802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.610227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.610679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.611130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.611575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.613857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.613857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.613857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.613857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.614553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.614553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.614731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.614936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.615296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.615633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.616156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.616467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.616682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.617010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.617589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.617918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.618493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.618829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.619179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.619504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.619825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.620146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.620466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.620784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.621100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.621416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.621772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.622117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.622444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.622778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.623146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.623474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.623796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.624116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.624437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.624799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.625121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.625436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.625756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.626078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.626433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.626704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.627175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.627503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.627837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.628156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.628476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.628792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.629110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.629440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.629782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.630107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.630429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.630745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.631063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.631634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.631954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.632292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.632624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.632852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.context [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),99fe71d5-840c-46a3-b7af-a004ed27ab79(cell1) {{(pid=124966) load_cells /opt/stack/nova/nova/context.py:464}} May 23 15:44:07.632987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.633314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.633632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.633961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.634271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.634583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.634897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.635234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.635546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.635870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.636186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.636501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.636824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.637106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:07.637227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.637663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.638018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.638335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.638665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.638803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:07.638989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.639333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.639572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:07.639723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.639959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.640270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.640577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.640894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.641206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.641519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.641693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:07.641846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.642129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.642441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.642748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.643060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.643914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.644252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.644345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO dbcounter [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Registered counter for database nova_cell0 May 23 15:44:07.644563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.644872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.645183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.645498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.645805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.646116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.646418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.646725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.647035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.648397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.648397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.648397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.648397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.648924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO dbcounter [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Registered counter for database nova_cell1 May 23 15:44:07.649024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.649024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.649194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.649503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.649814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.650123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.650590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.650802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.651077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.651418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.651728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.652091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.652414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.653010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.653308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.653627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.653966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.654303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.654616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.654926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.655264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.655598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.655913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.656268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.656645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_db.sqlalchemy.engines [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124966) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.657079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.657079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.657342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.657429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_db.sqlalchemy.engines [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124966) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.657516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.657786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.658101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.658415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.658728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.659662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.659950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.660270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.660620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.660946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.661270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.661588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.661979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.662259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.662539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.662864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.663234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.663594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.663844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.664160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.664505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.664850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writer thread running {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.664957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.665209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.665529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.665853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.666165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.666325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writer thread running {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.666488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.666813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.667192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.667507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.667812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.668121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.668466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.668792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.669122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.669429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.669735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.670042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.670352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.670675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.670988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.671327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.671632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.671948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.672253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.672567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.672868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.673173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.673479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.673784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.674088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.674393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.674702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.675005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.675362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.675668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.675973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.676279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.676582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.676888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.677193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.677528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.677860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.678175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.678500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.678813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.679160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.679471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.679796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.680119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.680437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.680746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.681077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.681392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.681710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.682024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.682335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.682683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.682995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.683353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.683682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.683994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.684319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.684637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.684961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.685282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.685596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.685930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.686240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.686550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.686871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.687206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.687538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.687844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.688193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.688522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.688856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.689276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.689494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.689827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.690177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.690523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.690873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.691227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.691556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.691868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.692182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:44:07.692425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124967) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:44:07.752329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO dbcounter [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Registered counter for database nova_api May 23 15:44:07.772408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_db.sqlalchemy.engines [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124967) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.780260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writer thread running {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.820270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO dbcounter [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Registered counter for database nova_cell0 May 23 15:44:07.827989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_db.sqlalchemy.engines [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124966) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.833042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writer thread running {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.837266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.context [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),99fe71d5-840c-46a3-b7af-a004ed27ab79(cell1) {{(pid=124967) load_cells /opt/stack/nova/nova/context.py:464}} May 23 15:44:07.841476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:07.843086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:07.843918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:07.846005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:07.848592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO dbcounter [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Registered counter for database nova_cell0 May 23 15:44:07.852833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO dbcounter [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Registered counter for database nova_cell1 May 23 15:44:07.859891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_db.sqlalchemy.engines [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124967) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.862574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_db.sqlalchemy.engines [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124967) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:07.867843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writer thread running {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:07.869975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writer thread running {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:08.025276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO dbcounter [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Registered counter for database nova_cell0 May 23 15:44:08.032692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_db.sqlalchemy.engines [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124967) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:44:08.037948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writer thread running {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:44:08.161377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: WARNING keystonemiddleware.auth_token [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:44:08.168879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: WARNING keystonemiddleware.auth_token [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:44:08.368552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: WARNING keystonemiddleware.auth_token [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:44:08.375888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: WARNING keystonemiddleware.auth_token [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:44:08.515991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: WARNING keystonemiddleware.auth_token [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:44:08.522269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: WARNING keystonemiddleware.auth_token [None req-b7d987c8-8165-4244-9888-46edf8afc7e9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:44:08.528261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f7591f99c0 pid: 124966 (default app) May 23 15:44:08.723548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: WARNING keystonemiddleware.auth_token [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 23 15:44:08.729797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: WARNING keystonemiddleware.auth_token [None req-bd19b644-c83f-42f4-b99e-33a8ac45e172 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 23 15:44:08.735272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f7591f99c0 pid: 124967 (default app) May 23 15:44:51.136240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:51.138585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:51.138585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:51.139703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:52.027396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:52.028484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:52.029451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:52.030754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:44:52.608978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd646aae-8be6-4e5e-8d92-092a6c22e1a5 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Action: 'create', calling method: .version_select at 0x7f47fb921ca0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-82130167", "metadata": {"Type": "work"}}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd646aae-8be6-4e5e-8d92-092a6c22e1a5 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 120 May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:44:52.625570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-82130167", "metadata": {"Type": "work"}}} May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:44:52.627290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-82130167', 'metadata': {'Type': 'work'}}}} May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise error May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi '#$%' May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:52.628459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 15:44:52.630213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 15:44:52.631632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 23 15:44:52.631632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:52.631632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd646aae-8be6-4e5e-8d92-092a6c22e1a5 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:52.632484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fd646aae-8be6-4e5e-8d92-092a6c22e1a5 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.502683 May 23 15:44:52.633376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1/1] 10.222.0.32 () {60 vars in 1155 bytes} [Mon May 23 15:44:51 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1505 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:44:52.830138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bd3aa8a8-42ed-4b0d-a026-4616d03215ea tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Action: 'create', calling method: .version_select at 0x7f47fb807f70>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1498758055"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:52.941179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6b594d26-440b-40bc-8325-601162c94ef8 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Action: 'create', calling method: .version_select at 0x7f47fb921ca0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1682829238", "metadata": {"Type": "work"}}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:55.173518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6b594d26-440b-40bc-8325-601162c94ef8 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 3.164107 May 23 15:44:55.175067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1/2] 10.222.0.32 () {60 vars in 1155 bytes} [Mon May 23 15:44:52 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 3167 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 23 15:44:55.180986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d41c725-8787-4e36-a145-511a80a295e8 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Action: 'create', calling method: .version_select at 0x7f47fb772b80>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1717245862", "metadata": {"Type": "work"}}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d41c725-8787-4e36-a145-511a80a295e8 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 120 May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:44:55.193649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1717245862", "metadata": {"Type": "work"}}} May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: May 23 15:44:55.194920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1717245862', 'metadata': {'Type': 'work'}}}} May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise error May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi '0' May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:55.195796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 15:44:55.196968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 15:44:55.198143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 23 15:44:55.198143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.202690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d41c725-8787-4e36-a145-511a80a295e8 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:55.204695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d41c725-8787-4e36-a145-511a80a295e8 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.027244 May 23 15:44:55.206358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2/3] 10.222.0.32 () {60 vars in 1155 bytes} [Mon May 23 15:44:55 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 31 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:44:55.223346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-874b3ade-a28d-4f37-b4c5-9ccbf86f7248 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb772b80>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:55.324831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-874b3ade-a28d-4f37-b4c5-9ccbf86f7248 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.114058 May 23 15:44:55.325912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 3/4] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:55 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:55.330869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bfc4953c-c978-4958-928c-4383717e7047 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Action: 'create', calling method: .version_select at 0x7f47fb783160>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-894346842", "metadata": {"Type": "work"}}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bfc4953c-c978-4958-928c-4383717e7047 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 118 May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:44:55.333320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-894346842", "metadata": {"Type": "work"}}} May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: May 23 15:44:55.334290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-894346842', 'metadata': {'Type': 'work'}}}} May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise error May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi On instance['volume']: May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi '' May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:55.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 23 15:44:55.336003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 23 15:44:55.336842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 23 15:44:55.336842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:44:55.336842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bfc4953c-c978-4958-928c-4383717e7047 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:55.336842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bfc4953c-c978-4958-928c-4383717e7047 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.007695 May 23 15:44:55.343757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 4/5] 10.222.0.32 () {60 vars in 1155 bytes} [Mon May 23 15:44:55 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:44:55.423758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1342741603", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "networks": [{"uuid": "24afdbb5-50df-4c78-bc6c-8d0c2d595a59"}]}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:55.588025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] validate_networks() for [('24afdbb5-50df-4c78-bc6c-8d0c2d595a59', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:44:56.152484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bd3aa8a8-42ed-4b0d-a026-4616d03215ea tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 3.512071 May 23 15:44:56.155277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2/6] 10.222.0.32 () {60 vars in 1154 bytes} [Mon May 23 15:44:52 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 3521 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 23 15:44:56.161042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a822e679-c924-45a7-b43d-0ff0bbb6b996 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Calling method '.version_select at 0x7f47fb739160>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:56.225364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-a822e679-c924-45a7-b43d-0ff0bbb6b996 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] HTTP exception thrown: Volume tempest-invalid-1360423270 could not be found. May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a822e679-c924-45a7-b43d-0ff0bbb6b996 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1360423270 HTTP/1.1 May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:44:56.233253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume tempest-invalid-1360423270 could not be found. May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'tempest-invalid-1360423270'} May 23 15:44:56.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1360423270 could not be found. (HTTP 404) (Request-ID: req-aa1d00c0-128e-4ee6-86af-76a4fac4dbbb) May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:44:56.238072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1360423270 could not be found. May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:44:56.241485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1360423270 could not be found. May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1360423270 could not be found. May 23 15:44:56.242656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.245398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a822e679-c924-45a7-b43d-0ff0bbb6b996 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 404 to user: Volume tempest-invalid-1360423270 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:56.250327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a822e679-c924-45a7-b43d-0ff0bbb6b996 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1360423270" status: 404 len: 99 microversion: 2.1 time: 0.089816 May 23 15:44:56.250978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 3/7] 10.222.0.32 () {58 vars in 1217 bytes} [Mon May 23 15:44:56 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1360423270 => generated 99 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) May 23 15:44:56.264065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89c76102-a8c1-42d9-b79f-e8201f2b3712 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb7393a0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:56.350902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89c76102-a8c1-42d9-b79f-e8201f2b3712 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 200 len: 345 microversion: 2.1 time: 0.093462 May 23 15:44:56.352199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 4/8] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:56 2022] GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 345 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:56.357504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5551910-e1db-4de2-b677-cede80358365 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003895 May 23 15:44:56.358465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 5/9] 10.222.0.32 () {58 vars in 1139 bytes} [Mon May 23 15:44:56 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 23 15:44:56.367224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9b6e6915-0253-4b6b-a967-34efaa847c00 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb763a60>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:56.453216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9b6e6915-0253-4b6b-a967-34efaa847c00 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.093595 May 23 15:44:56.453826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 6/10] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:56 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:56.459510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5f6820f6-2833-41c9-9a43-529ca103d474 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003739 May 23 15:44:56.465483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 7/11] 10.222.0.32 () {58 vars in 1136 bytes} [Mon May 23 15:44:56 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 11 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) May 23 15:44:56.485458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee25249f-c576-49d4-828d-5f82d33c3cd2 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Calling method '.version_select at 0x7f47fb707430>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:56.537301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-ee25249f-c576-49d4-828d-5f82d33c3cd2 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] HTTP exception thrown: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee25249f-c576-49d4-828d-5f82d33c3cd2 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/os-volumes/64132ef3-f278-4719-b60e-6d593797ad69 HTTP/1.1 May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:44:56.539877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '64132ef3-f278-4719-b60e-6d593797ad69'} May 23 15:44:56.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. (HTTP 404) (Request-ID: req-90068b8e-c23f-416b-b5da-4e17d030056e) May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.541576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:44:56.543227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 15:44:56.544107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee25249f-c576-49d4-828d-5f82d33c3cd2 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 404 to user: Volume 64132ef3-f278-4719-b60e-6d593797ad69 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee25249f-c576-49d4-828d-5f82d33c3cd2 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/64132ef3-f278-4719-b60e-6d593797ad69" status: 404 len: 109 microversion: 2.1 time: 0.065267 May 23 15:44:56.544965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 8/12] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:44:56 2022] DELETE /compute/v2.1/os-volumes/64132ef3-f278-4719-b60e-6d593797ad69 => generated 109 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:44:56.552647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74843ef8-3294-47b5-b6cb-7844e72b6012 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Calling method '.version_select at 0x7f47fb763ca0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:56.641726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-74843ef8-3294-47b5-b6cb-7844e72b6012 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] HTTP exception thrown: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74843ef8-3294-47b5-b6cb-7844e72b6012 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Request method failure captured: May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/os-volumes/1633eca4-7a6d-4a33-a8b3-de1b72e36b4c HTTP/1.1 May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6vzgqJ7gahKDes57vV3hnF91cjNajtDo0_rILDkMY_nI065I5djfyjYqyOACXVz3LCUP-rSDwYXjBl69xAIeIp9wrT-I4p9mFuL1Og7zvDKTQVyJEZUIYuUHf_KXpSG_rOCxESGnQV8QIIra99_1vZ52449WFFT3oxjVkGgGxoEONw May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:44:56.644209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e479d40ca57c46a1a4e1deb7a6b5376f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e479d40ca57c46a1a4e1deb7a6b5376f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e479d40ca57c46a1a4e1deb7a6b5376f May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesNegativeTest-1697681185 May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: fcb281f8083a49fbad88ff29fd06fe2a May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesNegativeTest-1697681185-project May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '1633eca4-7a6d-4a33-a8b3-de1b72e36b4c'} May 23 15:44:56.645204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. (HTTP 404) (Request-ID: req-b8fc47a7-b077-4021-9358-87ba40591f15) May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.645835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:44:56.646683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 15:44:56.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. May 23 15:44:56.648384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:44:56.650615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74843ef8-3294-47b5-b6cb-7844e72b6012 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] Returning 404 to user: Volume 1633eca4-7a6d-4a33-a8b3-de1b72e36b4c could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:44:56.652305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74843ef8-3294-47b5-b6cb-7844e72b6012 tempest-VolumesNegativeTest-1697681185 tempest-VolumesNegativeTest-1697681185-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/1633eca4-7a6d-4a33-a8b3-de1b72e36b4c" status: 404 len: 109 microversion: 2.1 time: 0.104898 May 23 15:44:56.653547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 9/13] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:56 2022] GET /compute/v2.1/os-volumes/1633eca4-7a6d-4a33-a8b3-de1b72e36b4c => generated 109 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:44:57.364568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b3a61d1f-5d6d-4a47-b798-fe5c42449c6f tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb739040>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:57.448391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b3a61d1f-5d6d-4a47-b798-fe5c42449c6f tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 200 len: 346 microversion: 2.1 time: 0.088136 May 23 15:44:57.448948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 10/14] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:57 2022] GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 346 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:44:57.463679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4f5e190d-eaee-43d9-9817-bf27f0a13818 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Action: 'create', calling method: .version_select at 0x7f47fb707430>, body: {"snapshot": {"volume_id": "d0e4ab3f-b949-493a-b14d-29c6ff62c0bb", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-192467167"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:44:57.661901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:44:57.662413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:44:57.662773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:44:57.663207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:44:57.663620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:44:57.687573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:57.688020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:57.688861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:44:57.689295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:44:57.732234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Getting quotas for project ddcee48ecf6d4132a40fb625f930c43f. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:44:57.740207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Getting quotas for user 6a7cf8b784d44b5ebf246c9541e781d5 and project ddcee48ecf6d4132a40fb625f930c43f. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:44:57.766051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:44:57.768961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:44:57.949539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60a01824-cc67-4c7d-975f-08851d04cf79 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.604306 May 23 15:44:57.950474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 5/15] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:44:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 2606 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:44:57.963677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ee674a1-919b-4ee9-9ad4-e2477bfd3354 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fae37ca0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.073142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ee674a1-919b-4ee9-9ad4-e2477bfd3354 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 354 microversion: 2.1 time: 0.119418 May 23 15:44:58.073814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 6/16] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:57 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 354 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:58.090069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa907ce1-a71b-4b33-9b7a-c6ca56722d7f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.090700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa907ce1-a71b-4b33-9b7a-c6ca56722d7f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:44:58.320445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4f5e190d-eaee-43d9-9817-bf27f0a13818 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.860482 May 23 15:44:58.321101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 11/17] 10.222.0.32 () {60 vars in 1161 bytes} [Mon May 23 15:44:57 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 862 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:44:58.327638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-35803b57-6563-4981-84c4-003848723373 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb7638b0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.417497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-35803b57-6563-4981-84c4-003848723373 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 354 microversion: 2.1 time: 0.093730 May 23 15:44:58.419321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 12/18] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 354 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:58.459771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cce94a11-ab85-4ceb-a67e-d74a9412e35d tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb807b80>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.557349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cce94a11-ab85-4ceb-a67e-d74a9412e35d tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.136774 May 23 15:44:58.558997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 13/19] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:58.564643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70f06acf-2f75-458c-8b0e-26a2a89e6460 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb7070d0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.684519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fa907ce1-a71b-4b33-9b7a-c6ca56722d7f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:44:58.738033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa907ce1-a71b-4b33-9b7a-c6ca56722d7f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1215 microversion: 2.1 time: 0.635644 May 23 15:44:58.738713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 7/20] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1215 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:44:58.747519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cc304bd1-2159-4353-a42d-23a0a6472ba0 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fae4fdc0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.863314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cc304bd1-2159-4353-a42d-23a0a6472ba0 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.120641 May 23 15:44:58.864133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 8/21] 10.222.0.32 () {58 vars in 1139 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:58.874852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06429d8b-f056-46aa-852d-9b0610f10408 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa5f5940>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:58.945861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70f06acf-2f75-458c-8b0e-26a2a89e6460 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 202 len: 0 microversion: 2.1 time: 0.385320 May 23 15:44:58.946153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 14/22] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:44:58 2022] DELETE /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 0 bytes in 387 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:44:58.947418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06429d8b-f056-46aa-852d-9b0610f10408 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.076268 May 23 15:44:58.948575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 9/23] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:44:58.964375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a13a111-cbff-41af-8b7c-995ef0216639 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb707ee0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:59.105668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a13a111-cbff-41af-8b7c-995ef0216639 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.154852 May 23 15:44:59.106255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 15/24] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:44:58 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:44:59.764988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3dc968b8-dc70-4cb7-9478-2fe07ad0576e tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:44:59.764988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3dc968b8-dc70-4cb7-9478-2fe07ad0576e tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:44:59.959696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-081e3b74-16e2-48a0-be5e-e53b25d55662 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb7070d0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:00.017447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-081e3b74-16e2-48a0-be5e-e53b25d55662 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.061707 May 23 15:45:00.018042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 16/25] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:44:59 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:00.120507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a619dead-3c87-4a42-9167-e5a49dc6e4d2 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb807b80>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:00.266393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a619dead-3c87-4a42-9167-e5a49dc6e4d2 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.152247 May 23 15:45:00.267461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 17/26] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:00 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:00.369755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3dc968b8-dc70-4cb7-9478-2fe07ad0576e tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:00.384646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3dc968b8-dc70-4cb7-9478-2fe07ad0576e tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1215 microversion: 2.1 time: 0.632632 May 23 15:45:00.386820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 10/27] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:44:59 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1215 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:01.028897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c56f4aaf-5a66-4ece-83e9-37d30e05ae37 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb7638b0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:01.122193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c56f4aaf-5a66-4ece-83e9-37d30e05ae37 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.097658 May 23 15:45:01.122787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 18/28] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:01 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:01.280289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9cfb5be-a2cc-443e-afea-9f39fdcabedf tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fa5f5160>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:01.403303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:01.404152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:01.420996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9cfb5be-a2cc-443e-afea-9f39fdcabedf tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.145290 May 23 15:45:01.421799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 11/29] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:01 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:01.437416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:01.439527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:01.441022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:01.442382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:01.716394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:01.801968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-59a72c2f-85d7-4fac-b45b-ea99793ce553 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1219 microversion: 2.1 time: 0.403634 May 23 15:45:01.807440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 19/30] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:01 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1219 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:02.134586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a198f945-f142-4c74-a532-b5074ed331f5 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fb73c160>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:02.185979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a198f945-f142-4c74-a532-b5074ed331f5 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.055212 May 23 15:45:02.188938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 12/31] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:02 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:02.434090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33974603-a598-4820-9e4a-977ae2a12836 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fb650280>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:02.544053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33974603-a598-4820-9e4a-977ae2a12836 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.113962 May 23 15:45:02.544923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 20/32] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:02 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:02.832821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:02.833437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:02.844782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:02.848232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:02.849245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:02.854877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:03.313278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:03.334407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b8b9db76-e630-45de-8493-23358ca042b3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1219 microversion: 2.1 time: 0.505599 May 23 15:45:03.335276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 13/33] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:02 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1219 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:03.350170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e8ab4160-c5ac-4499-ac45-03f97668e018 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa5e3e50>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:03.403225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e8ab4160-c5ac-4499-ac45-03f97668e018 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.057566 May 23 15:45:03.407290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 14/34] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:03 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:03.556214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e1dde48-dc5e-41f0-858c-fb70728fabdb tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fa5721f0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:03.659623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9b974425-d8fd-4f3f-be72-a14b76dda59f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: .version_select at 0x7f47fb663160>, body: {"keypair": {"name": "tempest-keypair-1265677059", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL4vkfSHuNSo5ZjJFwM/xrfaON2vDeUuFkA8s0vZxfPKpmsRkzEQ7PH5+6dp1lexQ9cYQLvWza9wOp6CEa7sw1bgntpYwgMGNq4MTqD30Shjabrr2aPUrshkHMweJawtSg=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:03.764284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e1dde48-dc5e-41f0-858c-fb70728fabdb tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.211755 May 23 15:45:03.769855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 15/35] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:03 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:03.786049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-9b974425-d8fd-4f3f-be72-a14b76dda59f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:03.807768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-9b974425-d8fd-4f3f-be72-a14b76dda59f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:04.058267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9b974425-d8fd-4f3f-be72-a14b76dda59f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.051565 May 23 15:45:04.059691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 21/36] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:45:03 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1054 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:45:04.343454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7233fb04-d620-4597-964a-a5ea94b8f2bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: .version_select at 0x7f47fa5e3550>, body: {"keypair": {"name": "tempest-keypair-1200855778", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKeo9sw9xSXBtMZ36/l7/YY1pp66KlkHH9ZJznbwpWrQ6lasv7iYDd0BPd3G1BrInoDfowRh6CpF9RFAGevueEGxwtqH5WP2JLEMQtmoCnZu3EMtlHgFKQTyuGM7sVu0BQ=="}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:04.354132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:04.367743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:04.370650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:04.371186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:04.371948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:04.372392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:04.461877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-7233fb04-d620-4597-964a-a5ea94b8f2bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:04.473560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-7233fb04-d620-4597-964a-a5ea94b8f2bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for user 6d689e7ca175463ab0c319260ba49d0d and project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:04.607583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:04.628820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78ab5f99-e37e-4e33-b9d4-197f08a0e217 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1219 microversion: 2.1 time: 0.274310 May 23 15:45:04.631319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 22/37] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:04 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1219 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:04.653040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74a45b01-db5e-4ee6-a790-32edfbcf9905 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fad71e50>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:04.682847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7233fb04-d620-4597-964a-a5ea94b8f2bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.791322 May 23 15:45:04.683495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 16/38] 10.222.0.32 () {62 vars in 1199 bytes} [Mon May 23 15:45:03 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 793 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 23 15:45:04.714078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74a45b01-db5e-4ee6-a790-32edfbcf9905 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.081347 May 23 15:45:04.715345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 23/39] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:04 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:04.779049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0db41e00-1409-42eb-a85b-4e8e6a06404c tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fa572c10>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:04.889394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0db41e00-1409-42eb-a85b-4e8e6a06404c tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 200 len: 353 microversion: 2.1 time: 0.114299 May 23 15:45:04.890136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 17/40] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:04 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 353 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:05.651914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:05.651914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:05.663467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:05.666092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:05.678331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:05.681026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:05.728780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c77aa0e1-00b0-4745-97bc-2dd84b24e9fb tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa572550>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:05.813602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c77aa0e1-00b0-4745-97bc-2dd84b24e9fb tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 287 microversion: 2.1 time: 0.090267 May 23 15:45:05.814219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 18/41] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:05 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 287 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:05.825445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae1aec9b-6259-42c3-8072-4bb6de6c488b tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa57d280>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:06.077373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:06.105136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-abca3adf-3556-41ca-a872-6ec9ae310002 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1267 microversion: 2.1 time: 0.460125 May 23 15:45:06.108071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 24/42] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:05 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1267 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:06.117339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eff49a17-143f-4c9e-b7b4-7f3d74b59ca7 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Calling method '.version_select at 0x7f47fad9f670>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:06.197483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae1aec9b-6259-42c3-8072-4bb6de6c488b tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "DELETE /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 202 len: 0 microversion: 2.1 time: 0.376186 May 23 15:45:06.199006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 19/43] 10.222.0.32 () {58 vars in 1253 bytes} [Mon May 23 15:45:05 2022] DELETE /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 0 bytes in 379 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:45:06.204109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-eff49a17-143f-4c9e-b7b4-7f3d74b59ca7 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] HTTP exception thrown: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eff49a17-143f-4c9e-b7b4-7f3d74b59ca7 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Request method failure captured: May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 HTTP/1.1 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6vza-Vke_UimySDyTwvTuTx9sTtL_MSs2TdMIo_3H_PpfhED__qgzYupKsQjSq20OMJngsx5i5p8v_rdxfQRW52gBKf_Co4d6jN9dpFEhnWBMe3lX80etza8UlcMav19IKj8A8PddDt99FZvrfqQk4SNnEyAQORhpsBdUa2BeuprsY May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 611f73594c3d4780b8bd4decdc3ec6d5 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesGetTestJSON-919072511 May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:45:06.206248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/611f73594c3d4780b8bd4decdc3ec6d5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/611f73594c3d4780b8bd4decdc3ec6d5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/611f73594c3d4780b8bd4decdc3ec6d5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_611f73594c3d4780b8bd4decdc3ec6d5", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesGetTestJSON-919072511 May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 611f73594c3d4780b8bd4decdc3ec6d5 May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesGetTestJSON-919072511 May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesGetTestJSON-919072511-project May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 3563f01a9a6a4a8ca3d35fc25644fd0d May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesGetTestJSON-919072511-project May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '0d7982bc-3e43-44cb-866f-0a9e0a52ac01'} May 23 15:45:06.207575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. (HTTP 404) (Request-ID: req-faacb912-936c-4bd1-9685-aa6ba0c92f90) May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:06.208493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:06.209609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 15:45:06.210728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eff49a17-143f-4c9e-b7b4-7f3d74b59ca7 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] Returning 404 to user: Volume 0d7982bc-3e43-44cb-866f-0a9e0a52ac01 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:45:06.215261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eff49a17-143f-4c9e-b7b4-7f3d74b59ca7 tempest-VolumesGetTestJSON-919072511 tempest-VolumesGetTestJSON-919072511-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01" status: 404 len: 109 microversion: 2.1 time: 0.097435 May 23 15:45:06.216081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a8fbc65-5c5f-4680-a900-9a06cdf4d822 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa591a60>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:06.223906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 25/44] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:06 2022] GET /compute/v2.1/os-volumes/0d7982bc-3e43-44cb-866f-0a9e0a52ac01 => generated 109 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:45:06.268417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a8fbc65-5c5f-4680-a900-9a06cdf4d822 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 200 len: 286 microversion: 2.1 time: 0.065346 May 23 15:45:06.269871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 20/45] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:06 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 286 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:07.130727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:07.135153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:07.155988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:07.156455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:07.159973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:07.159973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:07.287430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-594eae20-00c7-4dce-b617-49afaa28f370 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa57d790>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:07.339134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-594eae20-00c7-4dce-b617-49afaa28f370 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] HTTP exception thrown: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-594eae20-00c7-4dce-b617-49afaa28f370 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Request method failure captured: May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c HTTP/1.1 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v06pcOi1yjwPfvAETZDNFi49_xMQyPNKOJP3VFtXlVVtPsQEHs-uQcDtyMhvywhSRHthDIW8DswUn2yKcWvq0iJK_HUhaM1VWLUtmAZvWtfmoIqJTFG_1PBZnaCufDd4O1kg__5_Ogyms3fy7FBCagkmR11QCAb9Xn2xdtZvragx0 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:45:07.342716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_9c021bc3b0a641c885f56c761164758f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 01bbd426cdf144de99ee779535878b8c May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '5c16faa4-f098-47d4-b076-fcaa1ff86c7c'} May 23 15:45:07.343844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. (HTTP 404) (Request-ID: req-7f173378-f960-4b56-8dc1-615427938fb9) May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.344653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:07.345555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 23 15:45:07.347010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-594eae20-00c7-4dce-b617-49afaa28f370 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Returning 404 to user: Snapshot 5c16faa4-f098-47d4-b076-fcaa1ff86c7c could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-594eae20-00c7-4dce-b617-49afaa28f370 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c" status: 404 len: 111 microversion: 2.1 time: 0.061301 May 23 15:45:07.349281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 21/46] 10.222.0.32 () {58 vars in 1250 bytes} [Mon May 23 15:45:07 2022] GET /compute/v2.1/os-snapshots/5c16faa4-f098-47d4-b076-fcaa1ff86c7c => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:45:07.353746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9977570f-600c-443a-94e9-fa13b34eeb90 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fa5f51f0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:07.456448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:07.465290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9603cdc4-6135-43f1-9e66-6cb5985e8b63 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1273 microversion: 2.1 time: 0.343391 May 23 15:45:07.491436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 26/47] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:07 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1273 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:07.706373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:07.762050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9977570f-600c-443a-94e9-fa13b34eeb90 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 202 len: 0 microversion: 2.1 time: 0.412412 May 23 15:45:07.762983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 22/48] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:45:07 2022] DELETE /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 0 bytes in 414 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:45:07.772046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-12e33f01-49f1-4bd9-ae51-6b3919e10e28 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fad9f280>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:07.847345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-12e33f01-49f1-4bd9-ae51-6b3919e10e28 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 200 len: 345 microversion: 2.1 time: 0.079878 May 23 15:45:07.848523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 27/49] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:07 2022] GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 345 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:08.490661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:08.493014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:08.509331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:08.511538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:08.513728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:08.515487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:08.566331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-315608777", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:08.738855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] validate_networks() for [('b08a16eb-8530-462b-9842-46050f408ed8', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:45:08.739087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:08.752145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5c55ac6-add0-47ad-b2ca-c03be46e8630 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1273 microversion: 2.1 time: 0.269879 May 23 15:45:08.769731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 28/50] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:08 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1273 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:08.862572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ed0b4c4-4f3b-4a1b-ab6e-c54ea8d1370c tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fad9f670>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:08.938695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-1ed0b4c4-4f3b-4a1b-ab6e-c54ea8d1370c tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] HTTP exception thrown: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ed0b4c4-4f3b-4a1b-ab6e-c54ea8d1370c tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Request method failure captured: May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb HTTP/1.1 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v06pcOi1yjwPfvAETZDNFi49_xMQyPNKOJP3VFtXlVVtPsQEHs-uQcDtyMhvywhSRHthDIW8DswUn2yKcWvq0iJK_HUhaM1VWLUtmAZvWtfmoIqJTFG_1PBZnaCufDd4O1kg__5_Ogyms3fy7FBCagkmR11QCAb9Xn2xdtZvragx0 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:45:08.940659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_9c021bc3b0a641c885f56c761164758f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 01bbd426cdf144de99ee779535878b8c May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'd0e4ab3f-b949-493a-b14d-29c6ff62c0bb'} May 23 15:45:08.943773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. (HTTP 404) (Request-ID: req-c3a78f3a-23a2-403a-967b-52940c0988cd) May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:08.944549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:08.945522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 15:45:08.946575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ed0b4c4-4f3b-4a1b-ab6e-c54ea8d1370c tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Returning 404 to user: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ed0b4c4-4f3b-4a1b-ab6e-c54ea8d1370c tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 404 len: 109 microversion: 2.1 time: 0.085377 May 23 15:45:08.947571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 29/51] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:08 2022] GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 109 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:45:08.960361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fec2fd13-7c72-43fd-be0c-ee39bdacd46e tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fad7f550>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:09.060310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-fec2fd13-7c72-43fd-be0c-ee39bdacd46e tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] HTTP exception thrown: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fec2fd13-7c72-43fd-be0c-ee39bdacd46e tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Request method failure captured: May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb HTTP/1.1 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v06pcOi1yjwPfvAETZDNFi49_xMQyPNKOJP3VFtXlVVtPsQEHs-uQcDtyMhvywhSRHthDIW8DswUn2yKcWvq0iJK_HUhaM1VWLUtmAZvWtfmoIqJTFG_1PBZnaCufDd4O1kg__5_Ogyms3fy7FBCagkmR11QCAb9Xn2xdtZvragx0 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:45:09.062559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_9c021bc3b0a641c885f56c761164758f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 01bbd426cdf144de99ee779535878b8c May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'd0e4ab3f-b949-493a-b14d-29c6ff62c0bb'} May 23 15:45:09.063696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. (HTTP 404) (Request-ID: req-68e1ccc1-770d-4f7e-86b2-be29f5d3dd62) May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.064395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 23 15:45:09.065489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 23 15:45:09.068292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fec2fd13-7c72-43fd-be0c-ee39bdacd46e tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Returning 404 to user: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fec2fd13-7c72-43fd-be0c-ee39bdacd46e tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "DELETE /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 404 len: 109 microversion: 2.1 time: 0.108742 May 23 15:45:09.070741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 30/52] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:45:08 2022] DELETE /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 109 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:45:09.074442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-315b0e76-e592-464f-bbae-d024a476cbb1 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Calling method '.version_select at 0x7f47fad7f0d0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:09.174192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-315b0e76-e592-464f-bbae-d024a476cbb1 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] HTTP exception thrown: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-315b0e76-e592-464f-bbae-d024a476cbb1 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Request method failure captured: May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb HTTP/1.1 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v06pcOi1yjwPfvAETZDNFi49_xMQyPNKOJP3VFtXlVVtPsQEHs-uQcDtyMhvywhSRHthDIW8DswUn2yKcWvq0iJK_HUhaM1VWLUtmAZvWtfmoIqJTFG_1PBZnaCufDd4O1kg__5_Ogyms3fy7FBCagkmR11QCAb9Xn2xdtZvragx0 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:45:09.176363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/9c021bc3b0a641c885f56c761164758f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_9c021bc3b0a641c885f56c761164758f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 9c021bc3b0a641c885f56c761164758f May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-167013173 May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 01bbd426cdf144de99ee779535878b8c May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesSnapshotsTestJSON-167013173-project May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'd0e4ab3f-b949-493a-b14d-29c6ff62c0bb'} May 23 15:45:09.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. (HTTP 404) (Request-ID: req-cbf9578f-5bf6-4b55-a7f8-b58a4b4b95a0) May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.178322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi item = cinderclient( May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 23 15:45:09.179315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 23 15:45:09.180277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-315b0e76-e592-464f-bbae-d024a476cbb1 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] Returning 404 to user: Volume d0e4ab3f-b949-493a-b14d-29c6ff62c0bb could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:45:09.181257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-315b0e76-e592-464f-bbae-d024a476cbb1 tempest-VolumesSnapshotsTestJSON-167013173 tempest-VolumesSnapshotsTestJSON-167013173-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb" status: 404 len: 109 microversion: 2.1 time: 0.107356 May 23 15:45:09.184001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 31/53] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:09 2022] GET /compute/v2.1/os-volumes/d0e4ab3f-b949-493a-b14d-29c6ff62c0bb => generated 109 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:45:09.782521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:09.783342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:09.805206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:09.805668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:09.806462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:09.822531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:10.261437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:10.261437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e12f24e-8ff7-4f7e-9dd2-0c9f883323ae tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1273 microversion: 2.1 time: 0.467224 May 23 15:45:10.261437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 32/54] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:09 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1273 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:11.021205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:45:11.022475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:45:11.023231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:45:11.024036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:45:11.024761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:45:11.045562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:11.046467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:11.047591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:11.062750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:11.088988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:11.094962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for user 606ff49786a04aa5b677cbb9a57b216c and project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:11.109318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:45:11.112781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:45:11.282617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:11.284086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:11.293309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-555fd2a1-6534-424c-b806-92d219f299e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.936534 May 23 15:45:11.308136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:11.311756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:11.311756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:11.311756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:11.321464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 23/55] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:45:08 2022] POST /compute/v2.1/servers => generated 383 bytes in 2965 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:45:11.336774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-96f6adca-56fd-4458-8f3a-9c9a934c1532 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:11.343882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-96f6adca-56fd-4458-8f3a-9c9a934c1532 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:11.695556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96f6adca-56fd-4458-8f3a-9c9a934c1532 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:11.703475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-96f6adca-56fd-4458-8f3a-9c9a934c1532 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1221 microversion: 2.1 time: 0.378949 May 23 15:45:11.710531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 24/56] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:11 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1221 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:12.161343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-711769675", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5be72b67-de15-4ced-95f8-5cd9e93f3a7a"}], "security_groups": [{"name": "tempest-securitygroup--837345711"}], "key_name": "tempest-keypair-1200855778", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:12.284370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:12.319110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-80389476-1fa3-441f-b218-091b1adbe303 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 1.036410 May 23 15:45:12.337233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 33/57] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:11 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 1059 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:12.432887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-365922747", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:12.499172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] validate_networks() for [('5be72b67-de15-4ced-95f8-5cd9e93f3a7a', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:45:12.669820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] validate_networks() for [('b08a16eb-8530-462b-9842-46050f408ed8', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:45:13.842315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:14.430370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:45:14.433784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:45:14.433784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:45:14.436680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:45:14.437205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:45:14.474886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:14.475377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:14.476198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:14.476648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:14.586853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:14.602461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for user 6d689e7ca175463ab0c319260ba49d0d and project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:14.668719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:45:14.682185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:45:14.899790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0fcff19-5b2c-4933-97f2-a614085c006e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 2.743386 May 23 15:45:14.908609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 25/58] 10.222.0.32 () {62 vars in 1187 bytes} [Mon May 23 15:45:12 2022] POST /compute/v2.1/servers => generated 401 bytes in 2753 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) May 23 15:45:14.929401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:14.943754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:14.973945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:14.978737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:14.978737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:14.978737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:15.315723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:15.330667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21ead155-a1f5-4948-adc6-3006948daf80 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.417239 May 23 15:45:15.333741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 26/59] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:14 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:15.346836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:45:15.347393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:45:15.348225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1342103504", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7e7a8587-a97e-4c11-9c4a-682adeb9c236"}], "security_groups": [{"name": "tempest-securitygroup--1310058983"}], "key_name": "tempest-keypair-1265677059", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:15.349970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:45:15.353421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:45:15.353421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:45:15.409877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:15.413358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:15.418888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:15.420042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:15.575516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:15.578075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] validate_networks() for [('7e7a8587-a97e-4c11-9c4a-682adeb9c236', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:45:15.596428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for user 606ff49786a04aa5b677cbb9a57b216c and project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:15.624979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:45:15.628249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:45:15.916996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2b1d941-5451-4a89-bea2-56366067e7b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.493754 May 23 15:45:15.924304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 34/60] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:45:12 2022] POST /compute/v2.1/servers => generated 383 bytes in 3497 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 23 15:45:15.930136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:15.933769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:16.405945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:16.433490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:16.435918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:16.437748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1dec10ce-b220-48c3-b146-e454a3822913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1281 microversion: 2.20 time: 0.511721 May 23 15:45:16.443119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 35/61] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1281 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:16.452162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da7a7f70-a9c0-44a9-9096-fa0bbeda7ea8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:16.455986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da7a7f70-a9c0-44a9-9096-fa0bbeda7ea8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:17.023805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da7a7f70-a9c0-44a9-9096-fa0bbeda7ea8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:17.030502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da7a7f70-a9c0-44a9-9096-fa0bbeda7ea8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1221 microversion: 2.1 time: 0.585865 May 23 15:45:17.042061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 36/62] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:16 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1221 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:17.056498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:17.057858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:17.083689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:17.088428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:17.089249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:17.089697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:17.844388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:17.854873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca61bad4-fa7e-4ef5-a53d-3710775b2de7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.811538 May 23 15:45:17.855416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 37/63] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:17 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:17.988127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:45:17.988768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:45:17.989437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:45:17.989992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:45:17.990415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:45:18.066594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.077038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.078143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.087038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.175727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:45:18.187729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:45:18.273738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:45:18.276118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:45:18.473093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-feb4aa5c-8327-4947-a526-1bace1c361f0 req-6e39d94e-c640-4be6-8e0d-2ae2610b0626 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7be415a2-1a20-4f25-b51b-3b9b5e0f7a01", "tag": "752bf515-b2ab-46d2-9772-b42705c7834a"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:18.500610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-63d3f98a-32e9-4760-bf1f-e15a78a2751c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.163756 May 23 15:45:18.500610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 27/64] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:45:15 2022] POST /compute/v2.1/servers => generated 402 bytes in 3165 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:45:18.505625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03bb1ac7-1ce5-420a-abda-9fad40fa7bf5 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9ca70d0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:18.523785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-feb4aa5c-8327-4947-a526-1bace1c361f0 req-6e39d94e-c640-4be6-8e0d-2ae2610b0626 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.525707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-feb4aa5c-8327-4947-a526-1bace1c361f0 req-6e39d94e-c640-4be6-8e0d-2ae2610b0626 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.600727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-feb4aa5c-8327-4947-a526-1bace1c361f0 req-6e39d94e-c640-4be6-8e0d-2ae2610b0626 service nova] Creating event network-changed:752bf515-b2ab-46d2-9772-b42705c7834a for instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 on hv-cinder-83588 May 23 15:45:18.601359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-feb4aa5c-8327-4947-a526-1bace1c361f0 req-6e39d94e-c640-4be6-8e0d-2ae2610b0626 service nova] Acquired lock "compute-rpcapi-router" {{(pid=124966) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 23 15:45:18.611419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.611419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.622965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.623521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.627225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03bb1ac7-1ce5-420a-abda-9fad40fa7bf5 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/1d4a5536-26b9-4c39-8374-491acd7b7190" status: 200 len: 350 microversion: 2.1 time: 0.126428 May 23 15:45:18.631142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 28/65] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:18 2022] GET /compute/v2.1/os-volumes/1d4a5536-26b9-4c39-8374-491acd7b7190 => generated 350 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:18.639130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:18.639786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:18.668615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.compute.rpcapi [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 23 15:45:18.671995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] Releasing lock "compute-rpcapi-router" {{(pid=124966) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 23 15:45:18.679319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.682431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.688506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.689076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.726527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03eeac5c-36a5-4630-8c34-b8101cac9bcb None None] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.862529 May 23 15:45:18.728323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 38/66] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 866 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:18.734787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:18.737061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:18.757028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.757491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:18.758272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:18.758706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:19.172878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.177366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.186141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17579f52-230f-4770-9d40-2cac57f9f841 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1217 microversion: 2.1 time: 0.456011 May 23 15:45:19.187716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 39/67] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:18 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1217 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:19.189159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.193095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.197427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-236cf294-4300-46e2-8705-1037e6b5782d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1333 microversion: 2.20 time: 0.564700 May 23 15:45:19.214988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 29/68] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1333 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:19.222458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ce2d25c-1485-4725-9d49-3a988b8982d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:19.223567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:19.224118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0ce2d25c-1485-4725-9d49-3a988b8982d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:19.226016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:19.242020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=13 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:19.264674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:19.267753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:19.270206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:19.274991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:19.555004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0ce2d25c-1485-4725-9d49-3a988b8982d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.570833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0ce2d25c-1485-4725-9d49-3a988b8982d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1242 microversion: 2.1 time: 0.380733 May 23 15:45:19.571805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 40/69] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1242 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:19.692541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:19.706151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52a23a0d-14af-428f-982b-d3d31c4418cf tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.487019 May 23 15:45:19.706151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 30/70] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:19 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:20.206990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:20.214804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:20.215479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:20.216246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:20.238537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.238537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.238537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.238537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.253293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.254209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.256272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.257114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.574789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:20.604673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e0c55f7-4b23-47da-a38b-54a9214ae15d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1281 microversion: 2.1 time: 0.398891 May 23 15:45:20.605331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 41/71] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:20 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1281 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:20.620117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:20.623040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:20.660965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.661787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.662968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.664151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.727584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:20.759003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:20.761060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:20.762264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73c2e5d8-08f5-4756-af21-8e731228f4f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1351 microversion: 2.20 time: 0.551788 May 23 15:45:20.767971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 31/72] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1351 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:20.782817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:20.784306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:20.815347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.816357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.822220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:20.823335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:20.925645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:20.937572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-400fa11e-6a8c-449d-8438-1078e07bd2bf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1246 microversion: 2.1 time: 0.330877 May 23 15:45:20.938167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 42/73] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1246 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:21.625743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:21.628254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:21.631139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:21.640716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:21.641203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:21.643404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:21.643404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:21.972360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:21.988452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f5cf1b08-fdb0-482c-a18f-ff45d6638218 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1279 microversion: 2.1 time: 0.364968 May 23 15:45:21.997489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 43/74] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:21 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1279 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:22.005938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:22.007450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:22.015646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:22.023175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:22.024244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:22.025774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:22.027013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7f0398be-0a44-4a18-9676-cdab2917733d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 1.250238 May 23 15:45:22.027631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:22.029846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 32/75] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:20 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 1266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:22.046522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:22.050551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:22.075628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:22.076081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:22.076858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:22.077292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:22.296479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:22.310332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:22.313437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:22.317420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a66f8040-6aae-4417-a245-aa07c5dff68c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1339 microversion: 2.20 time: 0.318473 May 23 15:45:22.319771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 44/76] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1339 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:22.325624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:22.329254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f332f012-9e5f-4729-a6c7-d5cdb952594d tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9ca1700>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:22.341389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d604c94-6a40-43c3-a868-a98bb4f0323c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1294 microversion: 2.1 time: 0.310091 May 23 15:45:22.355223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 33/77] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1294 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:22.434667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f332f012-9e5f-4729-a6c7-d5cdb952594d tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/59ffcf9d-1179-4188-b0c4-8882562f2bef" status: 200 len: 351 microversion: 2.1 time: 0.113417 May 23 15:45:22.437894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 45/78] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:22 2022] GET /compute/v2.1/os-volumes/59ffcf9d-1179-4188-b0c4-8882562f2bef => generated 351 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:23.032102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:23.032873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:23.046626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:23.047549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:23.077484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.080224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.080898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.081545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.083356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.084231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.084902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.085553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.148477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=51 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:23.369681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:23.379784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-94577506-9de3-46eb-97dc-4f0b597aefd3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1279 microversion: 2.1 time: 0.360714 May 23 15:45:23.380374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 34/79] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:23 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1279 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:23.408621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:23.413224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:23.438113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.441641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.442769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.444372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.903333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:23.914849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-51ad8d7f-335d-4e9a-9d44-e643ecf79956 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.873827 May 23 15:45:23.919264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 46/80] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:23 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:23.930010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:23.931426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:23.958829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.960238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:23.962322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:23.965041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.251377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:24.280095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-728c8aeb-6c93-46fd-8ecc-b0473db97225 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1300 microversion: 2.1 time: 0.359385 May 23 15:45:24.288775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 47/81] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1300 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:24.429833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-779cfc28-74e6-4446-9514-a232a7cdee39 req-1b876994-4d7c-4c6a-b8fd-054cc50797a5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3831639c-bfc1-491d-a152-5b09cf9adf26", "tag": "2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:24.457150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:24.458216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-779cfc28-74e6-4446-9514-a232a7cdee39 req-1b876994-4d7c-4c6a-b8fd-054cc50797a5 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:24.458216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-779cfc28-74e6-4446-9514-a232a7cdee39 req-1b876994-4d7c-4c6a-b8fd-054cc50797a5 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.458472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:24.458472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:24.458472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ee330051-35ca-4d70-9cff-ce84dccccd73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 1.061290 May 23 15:45:24.458888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 35/82] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 1065 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:24.506630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:24.507386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:24.509742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-779cfc28-74e6-4446-9514-a232a7cdee39 req-1b876994-4d7c-4c6a-b8fd-054cc50797a5 service nova] Creating event network-changed:2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8 for instance 3831639c-bfc1-491d-a152-5b09cf9adf26 on hv-cinder-83588 May 23 15:45:24.534614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-779cfc28-74e6-4446-9514-a232a7cdee39 req-1b876994-4d7c-4c6a-b8fd-054cc50797a5 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.243584 May 23 15:45:24.583551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:24.583551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.583857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 48/83] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:24.583936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:24.583936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.674234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:24.682692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07fa6be1-5a50-4df2-8112-b53780f57f13 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1279 microversion: 2.1 time: 0.227556 May 23 15:45:24.685352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 36/84] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:24 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1279 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:24.932915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:24.933879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:24.943161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:24.943991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.945185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:24.945930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:24.952019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:25.198251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:25.206835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1d7c17e5-4a69-40c0-b007-a36010f22ba7 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.279301 May 23 15:45:25.221585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 49/85] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:24 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:25.304489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:25.305112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:25.316637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.317108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.318128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.318572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.474288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:25.474894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:25.482180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.482627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.483798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.484229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.638677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:25.649282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-349709cc-7972-4dc5-8756-3934666f7182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1300 microversion: 2.1 time: 0.348510 May 23 15:45:25.654971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 37/86] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1300 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:25.675982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a2df1dd-06ac-46cb-901e-bfbdf4ad805c tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9ca7700>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:25.769536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a2df1dd-06ac-46cb-901e-bfbdf4ad805c tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/755d0563-3412-4940-9e36-420da255b5db" status: 200 len: 350 microversion: 2.1 time: 0.098107 May 23 15:45:25.770101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 38/87] 10.222.0.32 () {58 vars in 1244 bytes} [Mon May 23 15:45:25 2022] GET /compute/v2.1/os-volumes/755d0563-3412-4940-9e36-420da255b5db => generated 350 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:25.775003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:25.775659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:25.784898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.789253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.790304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:25.790757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:25.838291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:25.848972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:25.850790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:25.860932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b418085-f702-44fb-a8a3-4df5aab4b1f5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.390816 May 23 15:45:25.874898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 50/88] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:25.885196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8fad704e-3ea0-460a-b24f-7560d3d72b14 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47facd33a0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:25.997767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8fad704e-3ea0-460a-b24f-7560d3d72b14 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.121309 May 23 15:45:25.998396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 51/89] 10.222.0.32 () {58 vars in 1133 bytes} [Mon May 23 15:45:25 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.012158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a625b827-2671-49fc-b812-801486ec52de tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9ca1280>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.079385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:26.096736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8871fb9-8f5c-4035-8dd2-81eac0df9a3d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.324498 May 23 15:45:26.098206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 39/90] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:25 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.112333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a625b827-2671-49fc-b812-801486ec52de tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.104601 May 23 15:45:26.113447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 52/91] 10.222.0.32 () {58 vars in 1148 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:26.131213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9c4620ff-0a07-470a-84e5-b2976ea1fb01 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9c72280>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.224937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.225632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:26.238520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9c4620ff-0a07-470a-84e5-b2976ea1fb01 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.107861 May 23 15:45:26.239981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 40/92] 10.222.0.32 () {58 vars in 1133 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.242094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.242613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.244232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.244836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.257730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b3c61e05-23af-4249-85d7-1ea58c5d02f6 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9c65820>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.378772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b3c61e05-23af-4249-85d7-1ea58c5d02f6 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.126778 May 23 15:45:26.378772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 41/93] 10.222.0.32 () {58 vars in 1166 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:45:26.389914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-58a38783-fe2a-42da-b7f8-8328b9ea448c tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9cc6b80>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.538175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-58a38783-fe2a-42da-b7f8-8328b9ea448c tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.152096 May 23 15:45:26.546211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 42/94] 10.222.0.32 () {58 vars in 1169 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:26.551573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-7aebfca9-0c8f-4906-bc46-aa935d34cac9 req-f620beb4-dd93-43f8-abd5-2af3bd4fc83f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:26.590884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-7aebfca9-0c8f-4906-bc46-aa935d34cac9 req-f620beb4-dd93-43f8-abd5-2af3bd4fc83f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.590884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-7aebfca9-0c8f-4906-bc46-aa935d34cac9 req-f620beb4-dd93-43f8-abd5-2af3bd4fc83f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.606790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:26.623128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b0cdbcd-fd94-4430-b07e-794022966fe8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.398396 May 23 15:45:26.623746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 53/95] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.630325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b7841fdd-b25e-4a2a-992d-5e192aa3a05f tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9c7b310>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.635912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-7aebfca9-0c8f-4906-bc46-aa935d34cac9 req-f620beb4-dd93-43f8-abd5-2af3bd4fc83f service nova] Creating event network-changed:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:45:26.636750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-7aebfca9-0c8f-4906-bc46-aa935d34cac9 req-f620beb4-dd93-43f8-abd5-2af3bd4fc83f service nova] Acquired lock "compute-rpcapi-router" {{(pid=124967) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 23 15:45:26.638351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.640078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.642122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.643305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.672383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.compute.rpcapi [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 23 15:45:26.676480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] Releasing lock "compute-rpcapi-router" {{(pid=124967) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 23 15:45:26.707782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9bce9187-6b95-4ef0-8abe-5272b4b8f813 None None] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.160256 May 23 15:45:26.708542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 43/96] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:26.722170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.722983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:26.742068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.742068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.742068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.742068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.768904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b7841fdd-b25e-4a2a-992d-5e192aa3a05f tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.143847 May 23 15:45:26.769784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 54/97] 10.222.0.32 () {58 vars in 1154 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.781407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-403dcc23-e1bd-4e70-b17b-4f345f991246 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9ca1310>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.866058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-403dcc23-e1bd-4e70-b17b-4f345f991246 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.088935 May 23 15:45:26.868655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 55/98] 10.222.0.32 () {58 vars in 1187 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:45:26.880163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:26.880839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:26.888298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.888840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.892366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:26.896016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:26.956654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:26.964859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2dd7b27b-f91a-486a-b1a5-21cf1506ffef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.255019 May 23 15:45:26.965475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 44/99] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:26.978916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a8c5795-cd72-4c97-bd02-3c1d2f835f26 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] Calling method '.version_select at 0x7f47f9c65a60>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:27.050708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:27.058985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:27.060775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:27.062282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-68ea7f94-212e-4dec-bc0c-5d4029e05630 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.186029 May 23 15:45:27.062840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 56/100] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:27.087375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a8c5795-cd72-4c97-bd02-3c1d2f835f26 tempest-VolumesTestJSON-138623693 tempest-VolumesTestJSON-138623693-project] 10.222.0.32 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.120439 May 23 15:45:27.087942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 45/101] 10.222.0.32 () {58 vars in 1154 bytes} [Mon May 23 15:45:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:27.114350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:27.116225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:27.122994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:27.123730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:27.124593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:27.125141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:27.636799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:27.653783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:27.659341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:27.659584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-405ac4eb-3851-4e6c-9329-65b2b3cb8008 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.543504 May 23 15:45:27.659584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 57/102] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:27 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:27.665077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:27.665848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:27.666907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:27.667588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:27.981834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:27.988091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:27.989208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:27.991085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-31e4fd23-05c8-40b1-8d5d-8186b3878f68 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.352025 May 23 15:45:28.000303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.002473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.004781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.005670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.008656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 46/103] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:27 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:28.080705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:28.081119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:28.101392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.103888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.108556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.109025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.214022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=34 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:28.428968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:28.438585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-51a07fe6-6675-4349-8991-d705f87b4f86 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.458100 May 23 15:45:28.440248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 58/104] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:28.525865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:28.537215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:28.540887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:28.540887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a86afa9-170f-42bc-b6b7-993cce39568b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.463521 May 23 15:45:28.540887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 47/105] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:28.676057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:28.677315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:28.692801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:28.692801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.692801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.692801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.693501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.724348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-52b4a2b0-2aef-4c9b-9b47-1dc485871eb4 req-78a43e96-ccc0-471f-85fa-6ad6ae83ae94 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6", "tag": "ee897199-ccbf-4d09-971b-4358bcee64ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:28.740736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-52b4a2b0-2aef-4c9b-9b47-1dc485871eb4 req-78a43e96-ccc0-471f-85fa-6ad6ae83ae94 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.741122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-52b4a2b0-2aef-4c9b-9b47-1dc485871eb4 req-78a43e96-ccc0-471f-85fa-6ad6ae83ae94 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.759278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-52b4a2b0-2aef-4c9b-9b47-1dc485871eb4 req-78a43e96-ccc0-471f-85fa-6ad6ae83ae94 service nova] Creating event network-changed:ee897199-ccbf-4d09-971b-4358bcee64ed for instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 on hv-cinder-83588 May 23 15:45:28.764633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-52b4a2b0-2aef-4c9b-9b47-1dc485871eb4 req-78a43e96-ccc0-471f-85fa-6ad6ae83ae94 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044552 May 23 15:45:28.765592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 48/106] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:28.822211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-67782d40-1f2b-4109-ab82-4c377303aa5f req-462668d8-fd44-4332-bd69-74e3d607834f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4c5a0e23-b645-46dd-8924-37f041a76f7c", "tag": "300c42fc-ece8-4ec8-987d-9dadf16db567"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:28.832891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-67782d40-1f2b-4109-ab82-4c377303aa5f req-462668d8-fd44-4332-bd69-74e3d607834f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:28.833547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-67782d40-1f2b-4109-ab82-4c377303aa5f req-462668d8-fd44-4332-bd69-74e3d607834f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:28.847044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-67782d40-1f2b-4109-ab82-4c377303aa5f req-462668d8-fd44-4332-bd69-74e3d607834f service nova] Creating event network-changed:300c42fc-ece8-4ec8-987d-9dadf16db567 for instance 4c5a0e23-b645-46dd-8924-37f041a76f7c on hv-cinder-83588 May 23 15:45:28.852968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-67782d40-1f2b-4109-ab82-4c377303aa5f req-462668d8-fd44-4332-bd69-74e3d607834f service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035996 May 23 15:45:28.854527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 49/107] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:28.933252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:28.944588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f247f0d-4c92-42a6-abcd-8a532930fd61 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.271290 May 23 15:45:28.955875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 59/108] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:28 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:29.011961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:29.012913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:29.020925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.021669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.022557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.023135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.190455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:29.198127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4d79d2d1-c759-4ee7-b9b9-b3d3a44fd964 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.192087 May 23 15:45:29.198696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 50/109] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:29 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:29.456273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:29.456884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:29.462865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.463346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.464111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.464544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.558351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:29.559363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=12 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:29.559494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:29.573272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.573736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.574503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.575067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.634855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:29.653447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d3bfe347-6a1a-4120-a671-55ce4456761b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.201375 May 23 15:45:29.654049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 60/110] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:29.743555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:29.753896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:29.755426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:29.757180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4c127aaa-381e-4764-b91a-b7e934c7fedd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.203137 May 23 15:45:29.757915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 51/111] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:29.965334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:29.966156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:29.974605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.975083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:29.976447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:29.977163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.122240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.129728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f38fe371-9be1-440f-a6af-dea1e8725a29 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.170778 May 23 15:45:30.130291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 61/112] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:29 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:30.217557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:30.217557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:30.223588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.224050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.224833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.225297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.375485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.382990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-20fd8481-fad8-4918-8cd5-76eedd19b7f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.172437 May 23 15:45:30.383596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 52/113] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:30 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:30.673104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:30.673104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:30.679501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.679501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.679501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.679501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.776357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:30.777557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:30.785690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.789490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.789490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:30.789490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:30.883314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.895210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4f23bdc4-7123-4721-80bd-b373ed147894 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.228771 May 23 15:45:30.896016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 62/114] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:30.971516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.980545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.982042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:30.983228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-51b34b05-b0c0-4015-b26d-e74d23e7ea5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.210983 May 23 15:45:30.983777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 53/115] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:31.148184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:31.149017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:31.155491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.156026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:31.157246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.157759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:31.332357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:31.340907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b3045b31-bea0-489f-941b-7d80fbad3607 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.197911 May 23 15:45:31.341487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 63/116] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:31 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:31.400681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:31.401349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:31.407895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.408986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:31.409872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.410397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:31.553294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:31.561207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abe40d1d-f0c6-4842-8b14-a17438a6700c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.164971 May 23 15:45:31.562024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 54/117] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:31 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:31.912586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:31.913334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:31.920437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.920993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:31.922013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:31.922645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.002924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:32.004221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:32.011007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.015496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.017091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.017091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.147695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:32.159888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3056b96b-70a4-4161-832d-e92242aed5f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.251180 May 23 15:45:32.160963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 64/118] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:32.208200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:32.217481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:32.219032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:32.220162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4e8c386b-6c86-4ba7-a41c-af4b3a894f32 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.221309 May 23 15:45:32.223518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 55/119] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:32.358361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:32.358963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:32.365735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.366226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.366972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.367428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.577952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:32.578574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:32.594782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.595283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:32.596650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:32.597093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:33.033164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:33.047594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34b89f29-2965-429f-bae2-c327daa471c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.693129 May 23 15:45:33.048186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 65/120] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:32 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:33.144962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:33.163566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c2b7a6fe-fe4f-4e14-b872-cecf31f4c435 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.589347 May 23 15:45:33.164153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 56/121] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:32 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:33.180714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:33.181437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:33.187966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:33.188507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:33.189420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:33.189924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:33.242120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:33.245100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:33.251987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:33.254025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:33.255272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:33.255893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:33.287939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:33.709014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:33.720096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4e9bcab6-47ca-4d81-9761-37a504a1ee67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.543286 May 23 15:45:33.730639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 66/122] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:34.001181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:34.015350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:34.020005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:34.024350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d74f5df6-fab6-4c09-840c-d96d7874ccb6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.787661 May 23 15:45:34.027686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 57/123] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:34.077732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:34.078484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:34.085779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.093444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.094438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.094992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.187456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:34.188792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:34.196115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.196900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.197918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.198552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.535831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:34.548548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-66d86e7a-d330-44ef-9e11-8b4efb010662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.475139 May 23 15:45:34.553036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 67/124] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:34 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:34.784979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:34.787667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-1f6d8fd4-d01c-4ad9-bcc5-5fc20e5eaef5 req-d08a679b-a596-4d77-9fd3-18e06159f871 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7be415a2-1a20-4f25-b51b-3b9b5e0f7a01", "name": "network-vif-plugged", "status": "completed", "tag": "752bf515-b2ab-46d2-9772-b42705c7834a"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:34.793536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de6f4c26-d3ca-4bcb-b21b-85646cdef359 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.613510 May 23 15:45:34.794234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 58/125] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:34 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:34.797403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-1f6d8fd4-d01c-4ad9-bcc5-5fc20e5eaef5 req-d08a679b-a596-4d77-9fd3-18e06159f871 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.801094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-1f6d8fd4-d01c-4ad9-bcc5-5fc20e5eaef5 req-d08a679b-a596-4d77-9fd3-18e06159f871 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.801210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:34.801210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:34.806844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.807344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.808127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:34.808581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:34.810765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-1f6d8fd4-d01c-4ad9-bcc5-5fc20e5eaef5 req-d08a679b-a596-4d77-9fd3-18e06159f871 service nova] Creating event network-vif-plugged:752bf515-b2ab-46d2-9772-b42705c7834a for instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 on hv-cinder-83588 May 23 15:45:34.816281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-1f6d8fd4-d01c-4ad9-bcc5-5fc20e5eaef5 req-d08a679b-a596-4d77-9fd3-18e06159f871 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.261906 May 23 15:45:34.817187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 68/126] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:34.998067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.006482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e82de7c4-de98-418d-9688-30628a7c9316 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.210972 May 23 15:45:35.007065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 59/127] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:35.054216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:35.055523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:35.066255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.067299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.068560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.069285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.263705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.272028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.273564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.274696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d50b303-7563-40f3-ab09-6257546be026 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.228581 May 23 15:45:35.275291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 69/128] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:35.570141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:35.570141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:35.576457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.576457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.577161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.583782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.776092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.784697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35ebefea-fb18-49b2-9548-f4e32141b2ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.220876 May 23 15:45:35.785533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 60/129] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:35 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:35.809739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:35.810357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:35.819837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.819837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.819837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:35.819837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:35.846506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:35.978790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:35.986306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e024224-6afc-4517-8144-a682f0b55200 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.180298 May 23 15:45:35.987151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 70/130] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:35 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:36.024893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:36.025509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:36.033311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.033311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.033311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.033572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.177620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:36.185565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39df709e-da45-479f-8e53-1e4133115a54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.164124 May 23 15:45:36.185779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 61/131] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:36.292664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:36.293495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:36.301930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.302567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.303529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.304114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.502137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:36.510662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:36.512489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:36.513620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-894e330a-8df7-4ff7-b499-68a971c27c73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.224889 May 23 15:45:36.514739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 71/132] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:36.802539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:36.806565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:36.815181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=8 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:36.815681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.816348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.817287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.817916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.834055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-21fc5c12-2f81-4519-8314-ea69f2d155e3 req-686607ad-4628-45dc-bdc8-1c5e1766551f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3831639c-bfc1-491d-a152-5b09cf9adf26", "name": "network-vif-plugged", "status": "completed", "tag": "2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:36.844053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-21fc5c12-2f81-4519-8314-ea69f2d155e3 req-686607ad-4628-45dc-bdc8-1c5e1766551f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:36.844930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-21fc5c12-2f81-4519-8314-ea69f2d155e3 req-686607ad-4628-45dc-bdc8-1c5e1766551f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:36.858946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-21fc5c12-2f81-4519-8314-ea69f2d155e3 req-686607ad-4628-45dc-bdc8-1c5e1766551f service nova] Creating event network-vif-plugged:2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8 for instance 3831639c-bfc1-491d-a152-5b09cf9adf26 on hv-cinder-83588 May 23 15:45:36.865277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-21fc5c12-2f81-4519-8314-ea69f2d155e3 req-686607ad-4628-45dc-bdc8-1c5e1766551f service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036083 May 23 15:45:36.866242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 72/133] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:36.999526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.006013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:37.007349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:37.007461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6d8bd038-e072-4add-8504-e62d14138d7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.210481 May 23 15:45:37.008035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 62/134] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:36 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:37.014300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.014748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.015544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.016407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.201632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.203135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:37.203955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:37.212075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ba71d1c3-e581-4ffc-a512-c6f78f79102f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.211237 May 23 15:45:37.212657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 73/135] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:36 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:37.222514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.223277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.224229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.224916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.531841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:37.532970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:37.540928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.541677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.542612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:37.543141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:37.574461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.582985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9afb05a3-3053-4678-a9e5-89335f892999 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.383733 May 23 15:45:37.583607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 63/136] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:37.697751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.712003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.714757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:37.717360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e82dd37-8c9b-46cb-922a-aa831615145e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.188955 May 23 15:45:37.718953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 74/137] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:38.024125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:38.024739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:38.031243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.031687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.032439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.032869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.221875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.230944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-56afb5ef-9f18-4674-89c8-6b0a19fe7043 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.210491 May 23 15:45:38.239299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 64/138] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:38 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:38.246964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:38.247660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:38.255036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.255440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.256067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.256500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.401908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.416852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f0317e44-8e01-46fa-87f6-a64fc9509f5c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.190595 May 23 15:45:38.417663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 75/139] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:38 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:38.600374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:38.601181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:38.608770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.609633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.611060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.611903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.735395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:38.736116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:38.741971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.742419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.745985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:38.745985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:38.747442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.755371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc7af2a0-e416-46c9-b1ea-87014f5ff450 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.159796 May 23 15:45:38.755942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 65/140] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:38.889400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.900124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.901681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:38.902812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-66d5d7ed-256b-49fc-a8ed-39faa210b55e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.171360 May 23 15:45:38.916753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 76/141] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:39.256637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:39.258308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:39.270301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.271054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.272404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.273172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.428457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:39.436552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af61137d-3dcc-4db6-ac0b-ad3ea4a9a213 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.190686 May 23 15:45:39.438315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 66/142] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:39 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:39.441448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:39.442682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:39.454794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.455732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.456579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.457188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.601971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:39.628270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f7b9ba3-82d2-46ea-8dd1-f6bbaf6d2e58 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.192189 May 23 15:45:39.628853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 77/143] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:39 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:39.773139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:39.773943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:39.780453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.781105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.782127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.782711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.928720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:39.930602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:39.932957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:39.937610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9fb0afc2-dcca-4b5a-a77f-18cee7c19dcf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.168295 May 23 15:45:39.938517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 67/144] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:39.953504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.953960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.954735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.955234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.963818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-fe3c56e8-46b2-4666-b813-e2af530b719e req-ecc1de7d-4959-4eb8-924c-2e612e640d51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "name": "network-vif-plugged", "status": "completed", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:39.978481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-fe3c56e8-46b2-4666-b813-e2af530b719e req-ecc1de7d-4959-4eb8-924c-2e612e640d51 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:39.978481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-fe3c56e8-46b2-4666-b813-e2af530b719e req-ecc1de7d-4959-4eb8-924c-2e612e640d51 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:39.994167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-fe3c56e8-46b2-4666-b813-e2af530b719e req-ecc1de7d-4959-4eb8-924c-2e612e640d51 service nova] Creating event network-vif-plugged:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:45:39.999996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-fe3c56e8-46b2-4666-b813-e2af530b719e req-ecc1de7d-4959-4eb8-924c-2e612e640d51 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042547 May 23 15:45:40.001144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 68/145] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:40.097923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:40.106114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:40.107631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:40.108736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-18eebc7e-7265-48dd-b329-6e9baaf926c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.188088 May 23 15:45:40.109289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 78/146] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:40.455321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:40.456030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:40.462326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.462779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:40.463591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.464024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:40.623405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:40.633001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ff71aa6-d31e-4924-aaa6-6eaa98fda5d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.181828 May 23 15:45:40.633576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 69/147] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:40 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:40.647735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:40.648407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:40.654533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.655069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:40.655871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.656300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:40.789243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:40.797625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9a0140ee-2cab-4825-afce-d4cb0ca7939d tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.153265 May 23 15:45:40.798208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 79/148] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:40 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:40.955578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:40.956188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:40.962269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.962721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:40.963525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:40.963967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.127994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:41.128603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:41.134647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.136211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.137174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.137643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.146843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.158417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-526a176d-6b4c-438f-b546-1bc57690b6c4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.206960 May 23 15:45:41.168825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 70/149] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:41.284168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.293035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.294620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.295817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-212073c4-253c-4444-9fec-d870910e3355 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.172377 May 23 15:45:41.296383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 80/150] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:41.649645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:41.650268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:41.656284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.656740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.657504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.657943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.791320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.799469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ece62c4-ab38-4a1b-a8a4-12d33d3a56a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.153193 May 23 15:45:41.800221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 71/151] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:41 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:41.814089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:41.814937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:41.821677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.822255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.823140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:41.823657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:41.955167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:41.962612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ee9a16a-6a58-47f7-8cb2-1677659b842f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.152382 May 23 15:45:41.963177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 81/152] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:41 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:42.175134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:42.175756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:42.185036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.188002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.188002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.188002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.313693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:42.314344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:42.320325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.320697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.321479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.321885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.356915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:42.364676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9c57efc9-6744-4b2a-8a27-9a578ea000a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.193466 May 23 15:45:42.365576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 72/153] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:42.503724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:42.515026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:42.516595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:42.518800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6c622a4-cd37-470b-baa4-b50845f2796d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.208766 May 23 15:45:42.519683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 82/154] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:42.816574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:42.817190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:42.823125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.823564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.824390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.824816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.962599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:42.970588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70f8dd32-f5b1-462c-9a80-2412dbedc99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.158193 May 23 15:45:42.971190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 73/155] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:42 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:42.979938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:42.980562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:42.986924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.987418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:42.988176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:42.988612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.119205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:43.127625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a3ce2aee-7c6c-4da1-9e46-b5155dd43be1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.150925 May 23 15:45:43.128259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 83/156] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:42 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:43.381903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:43.382523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:43.388581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.389027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.389790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.390232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.539852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:43.542239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:43.544875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:43.550457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:43.552429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5688f32f-67c5-4541-a822-b82218f309ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.172590 May 23 15:45:43.552429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 74/157] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:43.557011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.557464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.558404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.558834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.686537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:43.695198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:43.696694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:43.697790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-90e63905-3c2b-4955-88dc-1380b189203e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.163489 May 23 15:45:43.698332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 84/158] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:43.987541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:43.988295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:43.995119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.995514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:43.996276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:43.996711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.129394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.136964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-789ca0cc-091e-4444-8a97-c7783d195733 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.153895 May 23 15:45:44.137526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 75/159] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:43 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:44.145070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:44.145888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:44.152036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.152480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.153230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.153658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.287449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.295069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-668b51f6-da4e-4a94-b3c2-a1ef70f2d284 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.154290 May 23 15:45:44.295656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 85/160] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:44 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:44.567887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:44.568808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:44.575548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.576221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.577273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.577898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.716315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:44.718675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:44.723043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.723702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.724454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:44.724886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:44.731187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.739512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-999432d8-3f56-413c-9531-578db46f21d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.176552 May 23 15:45:44.740101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 76/161] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:44.897076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.904970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.906465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:44.907605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-18322cf2-aee3-4826-af77-323cca445694 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.195122 May 23 15:45:44.908681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 86/162] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:45.155235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:45.156068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:45.165153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.165847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.167653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.168347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.312391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:45.313255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:45.319775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.320360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.321335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.321768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.342837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:45.351395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6e4b67ef-d426-457c-ad6d-3be0721f1b8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.200219 May 23 15:45:45.352450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 77/163] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:45 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:45.465261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:45.468916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:45.475838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73891604-3685-437f-b11c-772c253012f9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.165503 May 23 15:45:45.476425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 87/164] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:45 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:45.759210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:45.760087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:45.766553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.771435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.775202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.775202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.847609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-c6b826b5-4ed9-4c13-a6c5-528a323faa09 req-fd48839e-c0c2-4f11-9988-db349051ed68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c5a0e23-b645-46dd-8924-37f041a76f7c", "name": "network-vif-plugged", "status": "completed", "tag": "300c42fc-ece8-4ec8-987d-9dadf16db567"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:45.858115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-c6b826b5-4ed9-4c13-a6c5-528a323faa09 req-fd48839e-c0c2-4f11-9988-db349051ed68 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.858561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-c6b826b5-4ed9-4c13-a6c5-528a323faa09 req-fd48839e-c0c2-4f11-9988-db349051ed68 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.871937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-c6b826b5-4ed9-4c13-a6c5-528a323faa09 req-fd48839e-c0c2-4f11-9988-db349051ed68 service nova] Creating event network-vif-plugged:300c42fc-ece8-4ec8-987d-9dadf16db567 for instance 4c5a0e23-b645-46dd-8924-37f041a76f7c on hv-cinder-83588 May 23 15:45:45.886222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-c6b826b5-4ed9-4c13-a6c5-528a323faa09 req-fd48839e-c0c2-4f11-9988-db349051ed68 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041723 May 23 15:45:45.887874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 88/165] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:45:45.930155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:45.931863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:45.940396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.947345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.947345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:45.947345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:45.957922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:45.972037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02436280-d0bd-4415-8c51-7250d941de84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.215618 May 23 15:45:45.973429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 78/166] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:46.241510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:46.263443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:46.264963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:46.266101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7280e2cb-2655-4c76-80bf-6113401ce67b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.340854 May 23 15:45:46.266953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 89/167] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:45:46.368926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:46.369864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:46.376158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.376641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:46.377414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.377852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:46.496759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:46.497524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:46.510565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.511040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:46.511955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.512387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:46.722219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:46.729333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:46.730763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-222f751a-139a-447b-81b1-192ad2327c12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.365890 May 23 15:45:46.741074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 79/168] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:46 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:46.747027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f1b8633-edf3-4564-a58d-42c4308c8715 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.255374 May 23 15:45:46.747609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 90/169] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:46 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:46.990226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:46.990846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:46.997755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.998205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:46.998979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:46.999441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.152504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:47.160289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f70e3fb1-fd96-4e26-bfb9-61937b84f03a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.173965 May 23 15:45:47.160899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 80/170] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:47.284433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:47.285041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:47.291429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.291871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.292632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.293051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:47.468113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:47.469958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:47.471151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-345c6c35-75ea-429a-87fd-44885cbfdb70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.190940 May 23 15:45:47.471701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 91/171] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:47.752591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:47.752591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:47.764246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:47.765668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:47.774584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.775201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.775935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.776743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.777324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.778354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:47.781122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:47.787480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.017046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.020360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.028654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-691a6fbf-e250-4b28-8886-d5590887ac74 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.268513 May 23 15:45:48.029557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 92/172] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:47 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:48.034923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-b0cb576f-01d5-4ab1-bcec-b7006c0dd8ff req-04e67690-b874-4999-aef8-e5d719f4ad40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6", "name": "network-vif-plugged", "status": "completed", "tag": "ee897199-ccbf-4d09-971b-4358bcee64ed"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:48.042050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-861728d7-0662-40d7-b794-34fd255fbfed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.298240 May 23 15:45:48.042984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 81/173] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:47 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:48.043847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-b0cb576f-01d5-4ab1-bcec-b7006c0dd8ff req-04e67690-b874-4999-aef8-e5d719f4ad40 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:48.044779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-b0cb576f-01d5-4ab1-bcec-b7006c0dd8ff req-04e67690-b874-4999-aef8-e5d719f4ad40 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.068031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-b0cb576f-01d5-4ab1-bcec-b7006c0dd8ff req-04e67690-b874-4999-aef8-e5d719f4ad40 service nova] Creating event network-vif-plugged:ee897199-ccbf-4d09-971b-4358bcee64ed for instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 on hv-cinder-83588 May 23 15:45:48.074736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-b0cb576f-01d5-4ab1-bcec-b7006c0dd8ff req-04e67690-b874-4999-aef8-e5d719f4ad40 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043290 May 23 15:45:48.075630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 93/174] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:48.177677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:48.178355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:48.184496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:48.185003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.185755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:48.186254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.466383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.474905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d226c3c1-3569-4661-be39-5e3a50b64d79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.301207 May 23 15:45:48.475513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 82/175] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:48.492414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:48.493565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:48.505783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:48.506581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.507605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:48.508288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:48.647063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.655178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.656750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:48.657905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7f73f85a-3174-4609-a6de-b509bf3fc54f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.171952 May 23 15:45:48.663210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 94/176] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:49.045931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:49.046568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:49.053846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.053953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.054639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.055077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.063438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:49.065970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:49.081072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.086519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.087537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.088004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.297434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.306253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e390dc3-b7f6-4d5e-a2c2-5c0d22279b9f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.264116 May 23 15:45:49.306825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 83/177] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:49 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:49.322434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.331354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b40959b-7474-414b-a0ad-cecae4e91e55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.272040 May 23 15:45:49.331354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 95/178] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:49 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:49.493057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:49.495060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:49.500015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.500465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.501227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.501723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.629888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.637600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e78de821-66cc-48d2-a91f-e6a49ce8673e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.149110 May 23 15:45:49.638805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 84/179] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:49.675706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:49.676318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:49.682472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.682789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.683646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:49.684001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:49.885873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.895548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.897373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:49.899075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-351ef0f9-74ae-4582-b661-4d7f54f62b4f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1406 microversion: 2.20 time: 0.226925 May 23 15:45:49.899889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 96/180] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1406 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:50.323375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:50.324188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:50.327216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:50.331386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.332074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.333051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.333634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.346952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:50.348958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:50.357424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.358514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.359536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.359979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.561837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:50.562686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:50.571060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ec319869-afbb-4086-9812-92bff7078e41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.228060 May 23 15:45:50.575887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9375a04-edf7-4c71-8110-bed7b9c3bd96 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1315 microversion: 2.1 time: 0.256102 May 23 15:45:50.576430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 97/181] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:50 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:50.576742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 85/182] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:50 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1315 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:50.654957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:50.655593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:50.661527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.662028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.662714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.663161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.793873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:50.801939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6ae0264e-a8ef-4b86-8c0e-033ee3736d07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.150785 May 23 15:45:50.802544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 86/183] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:50.917760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:50.918214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:50.920273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:50.925638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.926082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:50.926915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:50.927287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.072544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:51.080779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:51.084548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:51.084548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4c8ecc6-63d7-4048-a4a3-fd0b8418cf04 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1597 microversion: 2.20 time: 0.169737 May 23 15:45:51.084548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 98/184] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:45:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1597 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:45:51.102593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:51.103109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:51.109013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.109465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.110230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.110660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.237870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1677d394-5380-47c0-9954-cd31bef2095f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.139148 May 23 15:45:51.238459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 87/185] 10.222.0.32 () {60 vars in 1315 bytes} [Mon May 23 15:45:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-interface => generated 284 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:45:51.594621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:51.596182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:51.596816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:51.597596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:51.606871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.611145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.612681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.613629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.614066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.618195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.619079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.619986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.843026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:51.851733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-107d5416-7f97-4d7d-99ec-582d871969a9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1502 microversion: 2.1 time: 0.262782 May 23 15:45:51.852308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 88/186] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:51 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1502 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:51.887131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:51.888845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:51.889672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:51.896880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dca211c6-6c8b-4cc5-b887-75a7c3157836 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.312513 May 23 15:45:51.904803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 99/187] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:51 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:51.911126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.928795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:51.933457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:51.934083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:52.131227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:52.139597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bdceebff-0c7f-428f-a780-4315df1bc9bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1368 microversion: 2.1 time: 0.281706 May 23 15:45:52.140177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 89/188] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1368 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:52.891090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-9670e1f6-f648-4f28-8969-e5b564727bc8 req-57d05559-cb6c-4728-bc43-50da389d70f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:52.901729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-9670e1f6-f648-4f28-8969-e5b564727bc8 req-57d05559-cb6c-4728-bc43-50da389d70f7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:52.902416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-9670e1f6-f648-4f28-8969-e5b564727bc8 req-57d05559-cb6c-4728-bc43-50da389d70f7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:52.915439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-9670e1f6-f648-4f28-8969-e5b564727bc8 req-57d05559-cb6c-4728-bc43-50da389d70f7 service nova] Creating event network-changed:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:45:52.916024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:52.916794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:52.922509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-9670e1f6-f648-4f28-8969-e5b564727bc8 req-57d05559-cb6c-4728-bc43-50da389d70f7 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035047 May 23 15:45:52.927392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 100/189] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:52.937059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:52.937059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:52.937059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:52.937059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:53.118089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:53.127491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21ac2f4d-4197-4d5a-984f-87875ded862d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.215774 May 23 15:45:53.128072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 90/190] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:52 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:53.159160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:53.160138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:53.166827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:53.167319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:53.168091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:53.168910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:53.613737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:53.625946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bf9cbe62-e78f-4dae-9e8f-7305c8f06995 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1559 microversion: 2.1 time: 0.469195 May 23 15:45:53.632568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 101/191] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1559 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:53.644754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:53.645556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:53.656405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:53.657518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:53.658842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:53.660353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:53.809126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ff46bbe-0016-437e-9184-c21065428612 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.168394 May 23 15:45:53.809936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 91/192] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 15:45:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-interface => generated 285 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:54.153549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:54.154827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:54.164310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:54.164978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:54.165987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:54.166628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:54.807782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "460f7c5a-773d-4179-a6b3-b5880536be7e"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:54.809364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:54.816200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:54.816562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:54.817466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:54.817897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:54.820547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:45:54.904890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:54.923514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-61ee0ad4-7222-4385-af9d-0ab5bc566243 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.782171 May 23 15:45:54.924327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 102/193] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:54 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:55.944179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:55.945985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:55.953344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:55.954083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:55.955568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:55.956135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:56.172228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:56.180374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-46f7fa8e-1db9-4e5c-bd9b-44cccd844919 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1321 microversion: 2.1 time: 0.243924 May 23 15:45:56.181017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 103/194] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:55 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1321 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:56.187051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-46ffaf4d-1e4f-442c-bcef-fdbfbe73692e req-70f6a2ef-58d6-438e-b4f6-e3b74facc690 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4c5a0e23-b645-46dd-8924-37f041a76f7c", "tag": "300c42fc-ece8-4ec8-987d-9dadf16db567"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:56.198122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-46ffaf4d-1e4f-442c-bcef-fdbfbe73692e req-70f6a2ef-58d6-438e-b4f6-e3b74facc690 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:56.402149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-46ffaf4d-1e4f-442c-bcef-fdbfbe73692e req-70f6a2ef-58d6-438e-b4f6-e3b74facc690 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:56.402149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-46ffaf4d-1e4f-442c-bcef-fdbfbe73692e req-70f6a2ef-58d6-438e-b4f6-e3b74facc690 service nova] Creating event network-changed:300c42fc-ece8-4ec8-987d-9dadf16db567 for instance 4c5a0e23-b645-46dd-8924-37f041a76f7c on hv-cinder-83588 May 23 15:45:56.402149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-46ffaf4d-1e4f-442c-bcef-fdbfbe73692e req-70f6a2ef-58d6-438e-b4f6-e3b74facc690 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038110 May 23 15:45:56.402149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 104/195] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:45:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:45:56.498932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-05f1090f-acd0-4abe-8c12-c76e2952c95c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "POST /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.695874 May 23 15:45:56.500079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 92/196] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:45:54 2022] POST /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01/os-volume_attachments => generated 194 bytes in 1698 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:45:57.198187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:57.208033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:57.208033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:57.208033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:57.214034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:57.214802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:57.440173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:57.449896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96e2e023-9ea6-4561-8a5d-dc04be691372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1515 microversion: 2.1 time: 0.255775 May 23 15:45:57.450490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 105/197] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:57 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1515 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:45:57.492325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:45:57.493677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:57.526075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:57.526682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:57.538389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:57.539387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:57.583760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'flavor' on Instance uuid 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:45:58.105113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48209981-c8a5-45fc-8e26-42d23ea3d372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6/action" status: 200 len: 32 microversion: 2.1 time: 0.617617 May 23 15:45:58.110293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 93/198] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:45:57 2022] POST /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6/action => generated 32 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 15:45:58.115145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:58.115992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:58.124008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:58.124640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:58.125575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:58.126143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:58.278432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:58.291526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b285202c-74a6-478b-80e9-91502ac1f79b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.179215 May 23 15:45:58.291526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 106/199] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:58 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:45:59.309117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:45:59.309690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:45:59.316466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:59.316906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:59.317660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:45:59.318088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:45:59.466632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:45:59.475074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d358704b-b818-4dd7-932f-b9b081f2afec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.171086 May 23 15:45:59.475870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 94/200] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:45:59 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:00.496445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:00.497763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:00.505989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:00.506433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:00.507638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:00.508790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:00.513398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:00.663110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:00.675643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56b896be-c3ea-4726-92ca-3d1d2854351e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.185625 May 23 15:46:00.676224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 107/201] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:00 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:01.700245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:01.700990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:01.708207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:01.708695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:01.709469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:01.709909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:01.857720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:01.866070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-db5c0993-fd0e-4c0e-830c-e76b59d2ab60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.170300 May 23 15:46:01.866739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 95/202] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:01 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:02.891740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:02.893327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:02.904018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:02.904729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:02.905906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:02.906591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:03.052136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:03.061952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-050a142b-8102-41b0-a92c-d20c36d36664 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.177789 May 23 15:46:03.062530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 108/203] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:02 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:04.080546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:04.081285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:04.088434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:04.088801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:04.089428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:04.089864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:04.260067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:04.269560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d707fb66-0205-404e-ad2c-286db2a903b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.193343 May 23 15:46:04.270139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 96/204] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:04 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:05.288057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:05.288734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:05.294800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:05.295280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:05.296041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:05.296472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:05.445009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:05.456909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e074858-6169-47e4-8ff7-eebc14099027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.170364 May 23 15:46:05.456909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 109/205] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:05 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:06.471649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:06.472222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:06.478858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:06.479425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:06.480461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:06.480900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:06.614245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:06.623342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de0bcdee-8cd1-4a7d-a8a3-31d8d42d2851 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.155298 May 23 15:46:06.624109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 97/206] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:06 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:07.642833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:07.643472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:07.649582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:07.650022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:07.650777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:07.651236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:07.784197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:07.791915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c7a50aeb-43be-4cc9-b08d-d52469afb16c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.153651 May 23 15:46:07.792469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 110/207] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:07 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:08.263715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:08.264261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:08.271209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.271756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.272504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.272974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.301666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:46:08.698029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ee0fa7bb-b750-453f-b1aa-0087ea1c32e1 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "DELETE /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 204 len: 0 microversion: 2.1 time: 0.437749 May 23 15:46:08.698518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 98/208] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:46:08 2022] DELETE /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 15:46:08.707768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:08.708387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:08.714971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.715488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.716257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.716686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.810747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:08.810843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:08.818020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.818770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.820508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:08.820957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:08.873220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:08.882213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08d8a843-3a09-400f-9956-6493acef3464 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.179119 May 23 15:46:08.882808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 111/209] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:08 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:08.968421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:08.977922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5ad3bc10-c1db-45a2-b576-9a68cdf80c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.172088 May 23 15:46:08.978502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 99/210] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:08 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:09.899365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:09.899910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:09.905939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:09.906441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:09.907217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:09.908252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:09.994469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:09.995398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:10.002448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:10.011424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:10.011424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:10.011424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:10.037289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:10.045567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6347c96e-5399-4233-93e8-dcfad697650c tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.150282 May 23 15:46:10.046141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 112/211] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:09 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:10.138295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:10.146603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-057aae0f-bbba-477a-a580-766d0dfbad25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.155905 May 23 15:46:10.147214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 100/212] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:09 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:10.526354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1170288995", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:10.575291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] validate_networks() for None {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:46:10.856993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:46:10.857407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:46:10.857874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:46:10.858276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:46:10.858691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:46:10.868875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:10.869428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:10.870475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:10.870903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:10.887120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Getting quotas for project 485c8c6de3d34ae08491b570aa6c2520. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:46:10.890970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Getting quotas for user 1687d5d160e24bbaadab47bfd30e59f6 and project 485c8c6de3d34ae08491b570aa6c2520. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:46:10.902178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:46:10.904452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:46:11.063322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:11.063970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:11.070308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:11.070749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:11.071526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:11.071957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:11.195267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9ca7889-57b0-4499-95cd-c5ea18c98fd6 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.672527 May 23 15:46:11.195922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 113/213] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:46:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 674 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:46:11.200910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:11.201544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:11.207568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:11.208012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:11.208784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:11.209205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:11.212725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:11.222127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-16a58f5b-abb3-4254-b9be-5238de12936b tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.162578 May 23 15:46:11.222822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 101/214] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:11 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:11.228509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4375f227-1cbe-4f43-9f10-9d72cbef244e tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:11.229394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4375f227-1cbe-4f43-9f10-9d72cbef244e tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:11.315476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4375f227-1cbe-4f43-9f10-9d72cbef244e tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:11.322755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4375f227-1cbe-4f43-9f10-9d72cbef244e tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1215 microversion: 2.1 time: 0.090412 May 23 15:46:11.324151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 102/215] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:11 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1215 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:11.375358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:11.382739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-10762915-6e00-41b4-93ba-121355100902 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.185801 May 23 15:46:11.383335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 114/216] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:11 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:12.240893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:12.241511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:12.252374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:12.252825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:12.253659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:12.254093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:12.340197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a96284f1-7110-440b-bcc7-69dcb2391d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:12.340796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a96284f1-7110-440b-bcc7-69dcb2391d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:12.402586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a96284f1-7110-440b-bcc7-69dcb2391d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:12.409350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a96284f1-7110-440b-bcc7-69dcb2391d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1215 microversion: 2.1 time: 0.073371 May 23 15:46:12.409478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:12.409917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 115/217] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:12 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1215 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:12.417492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0f206e63-3437-4db1-8c2c-b459a625b8eb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.180876 May 23 15:46:12.418235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 103/218] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:12 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:12.424341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:12.424341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:12.428689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:12.429138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:12.429960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:12.430387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:12.562367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:12.566687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f979190-f45e-4038-9c6f-a01dc61c9bcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.149429 May 23 15:46:12.567285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 116/219] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:12 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:13.426867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:13.427509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:13.433803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:13.434243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.434592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:13.434687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.435439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.435881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.441076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.441520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.442273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.442699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.545249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:13.547939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:13.555243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-955d19d4-e86e-4f43-a3c4-95082ac7e7fb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1219 microversion: 2.1 time: 0.132212 May 23 15:46:13.562028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 104/220] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:13 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1219 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:13.586978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:13.587897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:13.594154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.594613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.597679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:13.599479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:13.622660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:13.637761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d2ba73dc-7615-4dc3-9b79-05d8eadc5523 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.207452 May 23 15:46:13.638448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 117/221] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:13 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:13.748072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:13.761200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a20d65a7-2a77-4f0c-b172-7a623539f2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.178123 May 23 15:46:13.761850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 105/222] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:13 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:14.576742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:14.577268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:14.584235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.584679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.585436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.585862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.646626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:14.654116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b0ddf195-6ff9-4e6f-87e4-03ceab3f9aeb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1219 microversion: 2.1 time: 0.081335 May 23 15:46:14.654749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 118/223] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:14 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1219 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:14.656811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:14.657503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:14.665172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.679930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.679930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.679930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.778359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:14.778997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:14.784943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.785382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.786147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:14.786572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:14.794560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:14.801926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3bd5f207-d38e-45d7-becc-b8085f0d9451 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.149152 May 23 15:46:14.802491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 106/224] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:14 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:14.917676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:14.925728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cb321cb0-39b1-47a2-8a98-1244f929428d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.151266 May 23 15:46:14.926522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 119/225] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:14 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:15.671409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:15.672079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:15.679150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.679603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.680363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.680796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.740600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:15.748387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abaf97a5-6362-4637-9b79-a7c7cdd67ad1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1211 microversion: 2.1 time: 0.080835 May 23 15:46:15.748953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 107/226] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:15 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1211 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:15.818996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:15.819636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:15.825809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.826222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.827036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.827561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.942059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:15.942678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:15.948859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.949311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.950379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:15.950978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:15.960235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:15.969949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8d609c9-899b-403f-97ad-9d784cca81f5 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.154882 May 23 15:46:15.971171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 120/227] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:15 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:16.089165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:16.097825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3726fd86-5c38-442a-9339-88d1999935cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.159511 May 23 15:46:16.098463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 108/228] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:15 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:16.765386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:16.765992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:16.772000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:16.772438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:16.773189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:16.773614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:16.834538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:16.842086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67a239ac-68c6-4eac-846a-9850af81d25b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.080636 May 23 15:46:16.842665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 121/229] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:16 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:16.987983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:16.988621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:16.996226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:16.996665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:16.997418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:16.997852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:17.120463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:17.121084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:17.127044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:17.127504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:17.128272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:17.128693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:17.130532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:17.140009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-65959e84-5d0c-49d8-beae-b12c753613cc tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.156262 May 23 15:46:17.140626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 109/230] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:16 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:17.269217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:17.276806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c071362b-4c1b-4362-a2f3-2198b4ebcc97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.162705 May 23 15:46:17.277369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 122/231] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:17 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:17.859811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:17.860815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:17.866391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:17.866839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:17.867627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:17.868057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:17.928915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:17.936744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9cbc9898-14cb-4068-9b1e-76937007aad9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.080781 May 23 15:46:17.937310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 110/232] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:17 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:18.156839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:18.157442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:18.163882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.164325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:18.165076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.165502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:18.289682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:18.293552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:18.294195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:18.297877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0db29037-b611-4274-a775-874d6b7cb0d3 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.144857 May 23 15:46:18.298960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 123/233] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:18 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:18.300996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.301452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:18.302226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.302670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:18.434213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:18.443299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-34497033-6270-4fbd-ae6e-1a30db6ac713 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.153440 May 23 15:46:18.443886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 111/234] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:18 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:18.953699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:18.954439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:18.960456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.960897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:18.961700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:18.962177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:19.022115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:19.029504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e7b2ae5f-5088-49af-a2e8-5c420e08b63f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.079737 May 23 15:46:19.030064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 124/235] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:18 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:19.314855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:19.315506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:19.322152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:19.322602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:19.323385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:19.323812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:19.452882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:19.460209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:19.460809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-98c951ca-160d-4334-90ab-aab0ae0055d9 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.149756 May 23 15:46:19.460911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:19.461378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 112/236] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:19 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:19.466805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:19.467285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:19.468048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:19.468478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:19.487650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:19.596486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:19.603931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d45583f4-4baa-4062-87b4-3a6462e69cba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.147862 May 23 15:46:19.604505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 125/237] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:19 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:20.046020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:20.046640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:20.052641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.053097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.053991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.054432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.116402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:20.124227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6ad9cdbd-ac91-4ef8-a5ce-3370990d30e4 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.082008 May 23 15:46:20.127265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 113/238] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:20 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:20.477987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:20.478600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:20.484611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.485053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.485852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.486217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.621707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:20.622186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:20.622524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:20.628983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.629432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.632864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3220ee86-168b-49d5-aa58-df0bb89ae7e8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.155777 May 23 15:46:20.632977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:20.633070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 126/239] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:20 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:20.633154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:20.764291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:20.772520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-74c77999-a8db-4754-b5e0-34698302e00a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.154895 May 23 15:46:20.773114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 114/240] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:20 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:21.141046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:21.141658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:21.144100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:21.148899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.155565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.155565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.155565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.209996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:21.217419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-37f8db63-e3c2-4f0b-89da-9c6ac803825b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.080210 May 23 15:46:21.217982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 127/241] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:21 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:21.647464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:21.647972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:21.649884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:21.651011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:21.655753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.656225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.657027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.657460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.789911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:21.790396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:21.790764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:21.792699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:21.797461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.797983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.798095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17bc07c3-4d17-421c-8d1c-4fed914772a2 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.154989 May 23 15:46:21.798579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 115/242] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:21 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:21.798977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:21.799502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:21.925705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:21.933045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02ce3346-3683-4f8c-afe5-8fee57a4bcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.147119 May 23 15:46:21.933615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 128/243] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:21 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:22.234270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:22.234890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:22.241975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.242426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:22.243217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.243653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:22.309668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:22.317054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e43dca5-be7b-49a6-835f-f11713653d08 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1267 microversion: 2.1 time: 0.087235 May 23 15:46:22.317642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 116/244] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:22 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1267 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:22.818298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:22.819682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:22.827810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=8 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:22.830904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.831750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:22.832889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.833815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:22.953255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:22.956009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:22.975130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.980719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:22.984176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:22.984733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:23.172760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:23.181887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-71c8f0a2-a25d-4f93-ba41-5c6039e70941 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1554 microversion: 2.1 time: 0.370524 May 23 15:46:23.182474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 129/245] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:22 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1554 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:23.199691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-5066a007-f5be-421c-b118-3c8d1589c239 req-d46580f1-88fb-4534-85c4-7402bb02ed37 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7be415a2-1a20-4f25-b51b-3b9b5e0f7a01", "name": "network-vif-deleted", "tag": "752bf515-b2ab-46d2-9772-b42705c7834a"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:23.206986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5066a007-f5be-421c-b118-3c8d1589c239 req-d46580f1-88fb-4534-85c4-7402bb02ed37 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:23.207549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5066a007-f5be-421c-b118-3c8d1589c239 req-d46580f1-88fb-4534-85c4-7402bb02ed37 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:23.255525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:23.269361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-306babb2-bed4-4afc-9ce0-cd8ab2e26d14 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.321905 May 23 15:46:23.270148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 117/246] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:22 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:23.334383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:23.339636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:23.353285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:23.355977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:23.362114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:23.362876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:23.388693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-5066a007-f5be-421c-b118-3c8d1589c239 req-d46580f1-88fb-4534-85c4-7402bb02ed37 service nova] Creating event network-vif-deleted:752bf515-b2ab-46d2-9772-b42705c7834a for instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 on hv-cinder-83588 May 23 15:46:23.394122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-5066a007-f5be-421c-b118-3c8d1589c239 req-d46580f1-88fb-4534-85c4-7402bb02ed37 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.204123 May 23 15:46:23.395117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 130/247] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:23.514546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:23.538377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-50775d6d-314a-4f4f-ac51-6c6d837db99f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1273 microversion: 2.1 time: 0.208375 May 23 15:46:23.548977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 118/248] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:23 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1273 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:24.201723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:24.207131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:24.212060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.213270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.214481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.215214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.286843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:24.288520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:24.296392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.296911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.297810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.298343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.310365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:24.324968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7d70a6cf-35fa-4628-8d37-22bf4e825dee tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1346 microversion: 2.1 time: 0.127845 May 23 15:46:24.325739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 131/249] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:24 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1346 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:24.496655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:24.505853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e1596800-d729-4a54-a1fa-d0250f8620c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.223182 May 23 15:46:24.506789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 119/250] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:24 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:24.559783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:24.560600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:24.567642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.568197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.569109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:24.569553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:24.720297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:24.728275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3b38e11a-ff4c-4982-80f9-7c4408861582 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.172268 May 23 15:46:24.729137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 132/251] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:24 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:25.111504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-e4a2fd0d-ee74-43e6-9d88-d4d196f46f6e req-f5d445b0-8617-4705-8632-5fba2c0abb69 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb4b831b-0839-44b4-809f-268bcbbf9462", "tag": "bd02ecbd-0cff-443a-a717-6e3927629b24"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:25.129676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-e4a2fd0d-ee74-43e6-9d88-d4d196f46f6e req-f5d445b0-8617-4705-8632-5fba2c0abb69 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.130117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-e4a2fd0d-ee74-43e6-9d88-d4d196f46f6e req-f5d445b0-8617-4705-8632-5fba2c0abb69 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.146365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-e4a2fd0d-ee74-43e6-9d88-d4d196f46f6e req-f5d445b0-8617-4705-8632-5fba2c0abb69 service nova] Creating event network-changed:bd02ecbd-0cff-443a-a717-6e3927629b24 for instance eb4b831b-0839-44b4-809f-268bcbbf9462 on hv-cinder-83588 May 23 15:46:25.154710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-e4a2fd0d-ee74-43e6-9d88-d4d196f46f6e req-f5d445b0-8617-4705-8632-5fba2c0abb69 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047302 May 23 15:46:25.155262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 120/252] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:46:25.342626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:25.343477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:25.350499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.350952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.351782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.352208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.384621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.384997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.385453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lazy-loading 'fault' on Instance uuid 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:46:25.444234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a7cf8b784d44b5ebf246c9541e781d5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ddcee48ecf6d4132a40fb625f930c43f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:25.452661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f82ffff-8465-47a2-a15a-5f3405a0d1bb tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 200 len: 1351 microversion: 2.1 time: 0.114414 May 23 15:46:25.453606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 133/253] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:25 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 1351 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:25.523774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:25.524477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:25.530668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.531157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.531916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.532355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.710371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:25.717926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-800f5972-e3f4-4ca0-9b90-b35a15bf517c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.198521 May 23 15:46:25.718516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 121/254] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:25 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:25.746077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:25.746772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:25.753586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.757744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.757744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:25.757744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:25.990815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:26.000036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2dffe3fb-24c9-46c8-9077-b9a79a95518c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.258011 May 23 15:46:26.000602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 134/255] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:25 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:26.472311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:26.473046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:26.479927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.480470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.481310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.481759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.500232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:26.500232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] HTTP exception thrown: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Request method failure captured: May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 HTTP/1.1 May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v2L7M-mzusf7okNEGGboaS4TqCv_5ztaHHme2Gg_gRLHGQA89rfe3Uy7shvXNJ6BcJ5B-axL1BWQEf2ssLFcXTBXU7fa2zvUOzvLOwLpP6lNsZqSMk4t__rlYw6xxz1o61lN_5SdhtIPOpneWlZoo2Y1dWLqVus19sur3D-LCmA2Q May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:46:26.505005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ddcee48ecf6d4132a40fb625f930c43f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 6a7cf8b784d44b5ebf246c9541e781d5 May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '7be415a2-1a20-4f25-b51b-3b9b5e0f7a01'} May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:46:26.507047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.508650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:46:26.509622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Returning 404 to user: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:46:26.509622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb89486d-789c-4de0-bfc6-45b785dd0662 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 404 len: 111 microversion: 2.1 time: 0.039101 May 23 15:46:26.509622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 122/256] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:26 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:46:26.629620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:26.630607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:26.637590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.638031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.639187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.640440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.697720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] HTTP exception thrown: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Request method failure captured: May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01/os-volume_attachments HTTP/1.1 May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v2L7M-mzusf7okNEGGboaS4TqCv_5ztaHHme2Gg_gRLHGQA89rfe3Uy7shvXNJ6BcJ5B-axL1BWQEf2ssLFcXTBXU7fa2zvUOzvLOwLpP6lNsZqSMk4t__rlYw6xxz1o61lN_5SdhtIPOpneWlZoo2Y1dWLqVus19sur3D-LCmA2Q May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:46:26.701860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ddcee48ecf6d4132a40fb625f930c43f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6a7cf8b784d44b5ebf246c9541e781d5 May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.704166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '7be415a2-1a20-4f25-b51b-3b9b5e0f7a01'} May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:46:26.704846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:26.706397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:26.709355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Returning 404 to user: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:46:26.709355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8afbf0b-d52c-4e48-8a43-38097b42a67f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.078213 May 23 15:46:26.709355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 135/257] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:46:26 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01/os-volume_attachments => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:46:26.736108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:26.737329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:26.744285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.744813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.746562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:26.747671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:26.926476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:26.934224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5e4b0521-618a-4231-9f52-4b761901fbbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.202869 May 23 15:46:26.935001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 123/258] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:26 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:27.021173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:27.021515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:27.028788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:27.028788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:27.029872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:27.030178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:27.190819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:27.201993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fd5d597a-5eea-401f-a9e7-63278442c67c tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.186407 May 23 15:46:27.203081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 136/259] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:27 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:27.951548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:27.957334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:27.959420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:27.960100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:27.961256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:27.962080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.094905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:28.102920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e76a0a1-c778-41c8-9a9a-93211c7504f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.155410 May 23 15:46:28.103531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 124/260] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:27 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:28.202574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:28.203122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:28.209320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.209766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.210522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.211029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.225049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] HTTP exception thrown: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.229354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:28.229354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Request method failure captured: May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 HTTP/1.1 May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v2L7M-mzusf7okNEGGboaS4TqCv_5ztaHHme2Gg_gRLHGQA89rfe3Uy7shvXNJ6BcJ5B-axL1BWQEf2ssLFcXTBXU7fa2zvUOzvLOwLpP6lNsZqSMk4t__rlYw6xxz1o61lN_5SdhtIPOpneWlZoo2Y1dWLqVus19sur3D-LCmA2Q May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:46:28.229571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ddcee48ecf6d4132a40fb625f930c43f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6a7cf8b784d44b5ebf246c9541e781d5 May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.230896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '7be415a2-1a20-4f25-b51b-3b9b5e0f7a01'} May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:46:28.231598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.232872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.234017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Returning 404 to user: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:46:28.234017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c14767e4-2fee-4886-9c56-e41f0f84f19f tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "DELETE /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 404 len: 111 microversion: 2.1 time: 0.029857 May 23 15:46:28.234017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 137/261] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:46:28 2022] DELETE /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:46:28.234219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.234423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.235379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.235978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.239370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:28.240005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] [instance: 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:28.246503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.246949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.247737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:28.248172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] HTTP exception thrown: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Request method failure captured: May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 HTTP/1.1 May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v2L7M-mzusf7okNEGGboaS4TqCv_5ztaHHme2Gg_gRLHGQA89rfe3Uy7shvXNJ6BcJ5B-axL1BWQEf2ssLFcXTBXU7fa2zvUOzvLOwLpP6lNsZqSMk4t__rlYw6xxz1o61lN_5SdhtIPOpneWlZoo2Y1dWLqVus19sur3D-LCmA2Q May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:46:28.265547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ddcee48ecf6d4132a40fb625f930c43f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ddcee48ecf6d4132a40fb625f930c43f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ddcee48ecf6d4132a40fb625f930c43f May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-DeleteServersTestJSON-1604489664 May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6a7cf8b784d44b5ebf246c9541e781d5 May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-DeleteServersTestJSON-1604489664-project May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.266805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '7be415a2-1a20-4f25-b51b-3b9b5e0f7a01'} May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:46:28.267489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. May 23 15:46:28.269040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:28.269972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] Returning 404 to user: Instance 7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:46:28.269972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-280b5b95-8433-4134-9156-458fd2fae2d8 tempest-DeleteServersTestJSON-1604489664 tempest-DeleteServersTestJSON-1604489664-project] 10.222.0.32 "GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01" status: 404 len: 111 microversion: 2.1 time: 0.031868 May 23 15:46:28.269972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 138/262] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:28 2022] GET /compute/v2.1/servers/7be415a2-1a20-4f25-b51b-3b9b5e0f7a01 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 15:46:28.488089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:28.493883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-621902cd-53a5-42e5-9455-c1e214e06110 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.271354 May 23 15:46:28.494879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 125/263] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:28 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:29.121482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:29.122096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:29.128886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:29.129291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:29.130077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:29.130600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:29.274530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:29.284609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af91603c-b834-411d-ad87-a0ed246ebfae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.167502 May 23 15:46:29.285200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 139/264] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:29 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:29.511639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:29.512259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:29.518770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:29.519266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:29.520315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:29.520763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:29.675016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:29.686018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c5c84163-cf22-47a5-9904-dc5eacd15c31 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.177474 May 23 15:46:29.686018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 126/265] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:29 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:30.302168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:30.304446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:30.311209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:30.311814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:30.313050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:30.313692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:30.474066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:30.481824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31228444-660a-4d29-9bd6-0ea07bcf6166 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.184909 May 23 15:46:30.482392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 140/266] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:30 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:30.702115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:30.702667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:30.709954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:30.710402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:30.711467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:30.712121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:30.865128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:30.872894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f0129d51-c76b-4151-abd1-832d6239d6cb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.175099 May 23 15:46:30.873452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 127/267] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:30 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:31.500438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:31.500438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:31.506062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:31.506614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:31.507763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:31.508053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:31.658881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:31.666647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ec0c286b-d28e-447b-b189-7c0d439c50a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.172742 May 23 15:46:31.667248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 141/268] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:31 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:31.893473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:31.894267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:31.901357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:31.902101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:31.902984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:31.903555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:32.047291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:32.055442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b21b4c34-82da-4a7c-abc0-e9ca738642ce tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.165870 May 23 15:46:32.056083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 128/269] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:31 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:32.684486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:32.685619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:32.694061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:32.694865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:32.696117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:32.696846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:32.870120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:32.881742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f2160eb-d137-44e2-8720-1ba176becf5b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.201658 May 23 15:46:32.882746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 142/270] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:32 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:33.072102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:33.072667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:33.080505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:33.080978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:33.081736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:33.083450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:33.328029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:33.337953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb8caf7e-4d2e-4818-bff7-8e11fe9f912f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.269587 May 23 15:46:33.344201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 129/271] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:33 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:33.701158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-575271061", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "networks": [{"uuid": "ad45c8b8-f9f9-4607-b7a6-9f5ec3d7435f"}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:33.771604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] validate_networks() for [('ad45c8b8-f9f9-4607-b7a6-9f5ec3d7435f', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:46:33.899636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:33.900257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:33.911141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:33.911606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:33.912759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:33.913141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.218897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:34.227724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1e0c7bd2-1e44-426f-876a-bb46d79112a2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.332133 May 23 15:46:34.237110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 130/272] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:33 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:34.360339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:34.362662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:34.373125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:34.373125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.373125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:34.373125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.652004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:46:34.654092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:46:34.654092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:46:34.654092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:46:34.655590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:46:34.666758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:34.667498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.668811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:34.669428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.675315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:34.685255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60e6a5d2-3a5a-42dc-bfa2-550a33cdb8f1 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.328642 May 23 15:46:34.685934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 131/273] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:34 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:34.695043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-d0f44f7b-bd58-4599-a3e3-72e88ab95538 req-f3ad3e71-98fe-4450-ac7d-0bac2d1aa03b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb4b831b-0839-44b4-809f-268bcbbf9462", "name": "network-vif-plugged", "status": "completed", "tag": "bd02ecbd-0cff-443a-a717-6e3927629b24"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:34.700115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d0f44f7b-bd58-4599-a3e3-72e88ab95538 req-f3ad3e71-98fe-4450-ac7d-0bac2d1aa03b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:34.700728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d0f44f7b-bd58-4599-a3e3-72e88ab95538 req-f3ad3e71-98fe-4450-ac7d-0bac2d1aa03b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:34.707313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Getting quotas for project b0c39de631f44292b3b0875386f17f3f. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:46:34.712806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Getting quotas for user bcaa4a7ea30242128c78eee9c2c73dfe and project b0c39de631f44292b3b0875386f17f3f. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:46:34.714681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-d0f44f7b-bd58-4599-a3e3-72e88ab95538 req-f3ad3e71-98fe-4450-ac7d-0bac2d1aa03b service nova] Creating event network-vif-plugged:bd02ecbd-0cff-443a-a717-6e3927629b24 for instance eb4b831b-0839-44b4-809f-268bcbbf9462 on hv-cinder-83588 May 23 15:46:34.719933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-d0f44f7b-bd58-4599-a3e3-72e88ab95538 req-f3ad3e71-98fe-4450-ac7d-0bac2d1aa03b service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032792 May 23 15:46:34.720560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 132/274] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:34.727448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:46:34.729753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:46:34.938536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cb530e0d-7966-4bfd-ad23-a17f29dfb87f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.241977 May 23 15:46:34.939562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 143/275] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:46:33 2022] POST /compute/v2.1/servers => generated 380 bytes in 1243 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:46:34.949870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-49e98780-547d-4178-b2e6-6f3260be390a tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:34.950881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-49e98780-547d-4178-b2e6-6f3260be390a tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:35.038659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-49e98780-547d-4178-b2e6-6f3260be390a tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:35.046417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-49e98780-547d-4178-b2e6-6f3260be390a tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1213 microversion: 2.1 time: 0.100593 May 23 15:46:35.046893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 133/276] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:34 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1213 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:35.244624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:35.245244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:35.252075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:35.252528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:35.253379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:35.253835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:35.496416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:35.504794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ecd293d-a84c-410a-8bce-a01ccc9b52c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.264153 May 23 15:46:35.505378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 144/277] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:35 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:35.702031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:35.702687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:35.710685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:35.710685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:35.711632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:35.712076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:35.917768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:35.926509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8f8d4298-254e-4847-92aa-483840f6df0a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.228296 May 23 15:46:35.927305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 134/278] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:35 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:36.064362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-498258ff-fec4-4ca5-b92f-b32d197d6542 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:36.065721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-498258ff-fec4-4ca5-b92f-b32d197d6542 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:36.126632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-498258ff-fec4-4ca5-b92f-b32d197d6542 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:36.134715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-498258ff-fec4-4ca5-b92f-b32d197d6542 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1213 microversion: 2.1 time: 0.074280 May 23 15:46:36.135666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 145/279] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:36 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1213 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:36.522224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:36.523325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:36.529598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:36.530330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:36.531345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:36.531992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:36.663600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:36.671749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d49e424-25bb-45b4-ab29-5bf48b781f09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.153903 May 23 15:46:36.672544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 135/280] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:36 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:36.943476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:36.944003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:36.950604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:36.951050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:36.951858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:36.952292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:36.956786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:37.100823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:37.110824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a64ed340-aa45-4aae-8779-b218314bbd8a tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.171225 May 23 15:46:37.121123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 146/281] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:36 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:37.154264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:37.154982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:37.162073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:37.162919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:37.164088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:37.164715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:37.236127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:37.247713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24a0d56f-5f44-410c-a568-75be85b2aaaf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1217 microversion: 2.1 time: 0.097352 May 23 15:46:37.248163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 136/282] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:37 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1217 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:37.692313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:37.693497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:37.708013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:37.708756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:37.709798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:37.710585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:37.888367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:37.901376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7f749dad-27a0-4ac0-a5cd-cc0a3aaae279 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.214621 May 23 15:46:37.910547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 147/283] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:37 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:38.133997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:38.134881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:38.144243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.144695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:38.145469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.145905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:38.157702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4ddbad9-2ada-4fb3-a23c-a555c2b74095 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: .version_select at 0x7f47f9a4b670>, body: {"keypair": {"name": "tempest-keypair-1113968782", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBENP59Mb5ml1L8xa1If2tFPNFMXRybylGmLR6kuqdA7MqqXTgvjSte738nOCEyCZO2eNth/uX7ncMJoulpppaDTdwZ24HYFwqVRbtbyBcXo+K1WohGlr4KEfQeEdwp4NzQ=="}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:38.174127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-a4ddbad9-2ada-4fb3-a23c-a555c2b74095 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for project e80aa9835e9845e798b1c8346de141e3. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:46:38.178867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-a4ddbad9-2ada-4fb3-a23c-a555c2b74095 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for user 8b0a82159c6343d28546ec7c775b8791 and project e80aa9835e9845e798b1c8346de141e3. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:46:38.288128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4ddbad9-2ada-4fb3-a23c-a555c2b74095 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.262205 May 23 15:46:38.289363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 137/284] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:46:38 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:38.294890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:38.296841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:38.303408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:38.309355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.309355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:38.309355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.309975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:38.312814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-095ebbe0-079b-4c60-8d90-caf76417a5a9 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.186173 May 23 15:46:38.322589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 148/285] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:38 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:38.379178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:38.386955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af43058d-b80d-4023-b050-0ac1867c7a4f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1217 microversion: 2.1 time: 0.096285 May 23 15:46:38.387762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 138/286] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:38 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1217 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:38.918051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:38.921876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:38.929147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.929602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:38.930379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:38.930817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:39.085106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:39.098205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03059cf0-1249-4401-ba80-4be8feb26029 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.182333 May 23 15:46:39.099244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 149/287] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:38 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:39.330918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:39.331632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:39.338460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:39.338999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:39.339976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:39.340486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:39.405362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:39.405986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:39.415908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:39.415908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:39.415908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:39.416305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:39.529074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:39.537927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8da133b-158d-4630-b79f-4c4be631f3ab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1209 microversion: 2.1 time: 0.137421 May 23 15:46:39.547689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 150/288] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:39 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1209 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:39.563626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:39.571783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-82926cf5-1ca1-42c5-82f1-c9e970b0fceb tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.245620 May 23 15:46:39.572353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 139/289] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:39 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:40.117007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:40.117641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:40.128877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.128877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.128877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.128877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.267007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:40.275205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0dd3688d-e5ab-4a76-b226-73f930e3b067 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.162209 May 23 15:46:40.275795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 151/290] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:40 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:40.555937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:40.556593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:40.570321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.571069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.572139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.572738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.589014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:40.596486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:40.599252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.599925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.600885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:40.601469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:40.643310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:40.651251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5da06f07-89f0-43f0-b2c7-fe5e82ed40b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1209 microversion: 2.1 time: 0.099266 May 23 15:46:40.652438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 140/291] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:40 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1209 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:40.761507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:40.771178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0befa23-1abb-4ff2-a4fe-a8ec5767b0ee tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.185055 May 23 15:46:40.771606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 152/292] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:40 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:41.093317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-699906610", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ec55f89b-6a09-430a-bad3-b52794f9cc56"}], "security_groups": [{"name": "tempest-securitygroup--1972088159"}], "key_name": "tempest-keypair-1113968782", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:41.205845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] validate_networks() for [('ec55f89b-6a09-430a-bad3-b52794f9cc56', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:46:41.293008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:41.293614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:41.300006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.300451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.301215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.301691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.551514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:41.563324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bfaca9ff-3eda-4c9c-bf00-134c9231599d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.274344 May 23 15:46:41.565131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 153/293] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:41 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:41.669106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:41.670638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:41.677861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.678521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.679649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.680074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.687741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:46:41.687741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:46:41.688301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:46:41.688578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:46:41.689798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:46:41.699842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.700408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.701392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.701939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.729373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for project e80aa9835e9845e798b1c8346de141e3. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:46:41.737277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for user 8b0a82159c6343d28546ec7c775b8791 and project e80aa9835e9845e798b1c8346de141e3. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:46:41.750439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:46:41.754351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:46:41.760778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:41.772456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3db205d3-8a1a-455b-a49b-f4cf9dfff0fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1209 microversion: 2.1 time: 0.106877 May 23 15:46:41.773383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 154/294] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:41 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1209 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:41.788230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:41.789193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:41.795795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.796327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.797111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:41.797540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:41.918853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8752ad46-3b23-4b1e-8100-e005588fc959 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.830055 May 23 15:46:41.920058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 141/295] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:46:41 2022] POST /compute/v2.1/servers => generated 402 bytes in 832 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:46:41.930146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-27543901-9eb1-453c-b7a2-b27193a98a52 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:41.930869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-27543901-9eb1-453c-b7a2-b27193a98a52 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:41.986925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:41.999366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b2232e6f-84c9-4875-9297-e555f2207cc7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.215519 May 23 15:46:42.012700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-27543901-9eb1-453c-b7a2-b27193a98a52 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:42.014195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 155/296] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:41 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:42.022648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-27543901-9eb1-453c-b7a2-b27193a98a52 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1237 microversion: 2.1 time: 0.096185 May 23 15:46:42.023854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 142/297] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1237 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:42.582354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:42.582980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:42.593695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:42.594146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:42.594926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:42.595425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:42.727495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:42.735266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a023605-32eb-42d5-8a4f-25d853266eca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.157397 May 23 15:46:42.735828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 156/298] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:42 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:42.790727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:42.791370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:42.797915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:42.798359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:42.799142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:42.799577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:42.863247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:42.865576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:42.871632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f64da6e-01fa-40ce-9340-dc52c30ad2fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1209 microversion: 2.1 time: 0.084847 May 23 15:46:42.872201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 143/299] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:42 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1209 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:43.019731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:43.020500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:43.026610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.027058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.027852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.028290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.040498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:43.041156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:43.047731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.048187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.051692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.051692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.128991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:43.136906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f1fcafd6-7369-4073-8836-28f2134b82ca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1241 microversion: 2.1 time: 0.100821 May 23 15:46:43.137350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 144/300] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1241 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:43.196614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:43.205733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a8f4180-4954-41d3-a23c-dd11c45c6e66 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.189609 May 23 15:46:43.206228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 157/301] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:43 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:43.752480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:43.753104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:43.759318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.759779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.760557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.760998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.887071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:43.888947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:43.889645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:43.894952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9f7793c5-92a2-45ca-9dad-b6a82a44f014 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.146279 May 23 15:46:43.895580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 145/302] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:43 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:43.896075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.896533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.897366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:43.897804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:43.958926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:43.966642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ba14c52-cec3-4810-b2eb-8a2be5aa94cf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1265 microversion: 2.1 time: 0.081582 May 23 15:46:43.967221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 158/303] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:43 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1265 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:44.154936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:44.155540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:44.162343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.162795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.163588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.164029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.228338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:44.229538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:44.233922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:44.238417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.239058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.240256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.240694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.246805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95003e72-8c6a-455f-885f-395ee4941faa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1233 microversion: 2.1 time: 0.093994 May 23 15:46:44.247748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 146/304] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1233 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:44.599120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:44.606441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03936cca-9577-4027-8e4f-870f1ca1e98b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.382022 May 23 15:46:44.607003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 159/305] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:44 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:44.912639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:44.913255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:44.919899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.920349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.926482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.926482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.985890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:44.987827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:44.992576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:44.997926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:44.998489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:44.999379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:45.000070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:45.082507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:45.086200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:45.090045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bc55b653-cc81-4507-b9fb-338016ebeed3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1265 microversion: 2.1 time: 0.110185 May 23 15:46:45.090623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 160/306] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:44 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1265 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:45.093909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ec45d120-1ef2-4da3-bac8-6c28d3b59a83 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1521 microversion: 2.1 time: 0.185353 May 23 15:46:45.094996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 147/307] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:44 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:45.267695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:45.268320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:45.274273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:45.274818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:45.275526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:45.275964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:45.336608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:45.343986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f097e440-e7b8-4253-b114-d22b4d7beb21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1289 microversion: 2.1 time: 0.080666 May 23 15:46:45.344551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 161/308] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1289 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:45.623419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:45.624036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:45.630031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:45.630489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:45.631284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:45.631724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:45.761534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:45.769726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4b653fe-7bf6-46c4-8429-d29f095a1c1b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1315 microversion: 2.1 time: 0.150465 May 23 15:46:45.770281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 148/309] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:45 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:46.106737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:46.107415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:46.110439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:46.111195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:46.113723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.114169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.114937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.115392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.117975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.118438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.119243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.119848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.132629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=8 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:46.190138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:46.193665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:46.198056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e82f5ffa-3245-431d-9114-5454db305d47 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1265 microversion: 2.1 time: 0.095393 May 23 15:46:46.198611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 162/310] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:46 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1265 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:46.278903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:46.287450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-175da65d-444d-4145-915d-1b570a7592e6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1501 microversion: 2.1 time: 0.180691 May 23 15:46:46.288029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 149/311] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:46 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1501 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:46.303775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:46.304384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:46.311513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.311959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.312716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.313149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.360231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:46.360857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:46.364357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:46.367984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.368440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.369271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.369690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.451537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:46.458648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a12c9d7-58b7-46b1-ad18-af60a0f3d61b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1289 microversion: 2.1 time: 0.102278 May 23 15:46:46.459277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 150/312] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1289 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:46.471480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:46.480741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de99cbf3-0a42-4693-9d42-2356e5c8b398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1515 microversion: 2.1 time: 0.180789 May 23 15:46:46.481336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 163/313] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:46 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1515 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:46.786567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:46.787090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:46.793631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.794076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.799350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:46.799350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:46.939968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:46.955021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f6bd0379-3f8d-4fbf-a792-5464fa9c9d63 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1464 microversion: 2.1 time: 0.173110 May 23 15:46:46.955655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 151/314] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:46 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1464 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:47.013531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:47.014033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:47.020999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.021517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.024985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.024985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.053668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:46:47.216901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:47.217575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:47.235482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.235482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.235482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.235482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.356433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:47.386223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d5277cf5-3c45-4a98-8ff8-8c410688d4a3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1273 microversion: 2.1 time: 0.174160 May 23 15:46:47.400085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 152/315] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:47 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1273 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:47.475057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:47.475811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:47.482528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.485825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.488324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.488916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.567973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:47.576456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0672afc2-531b-4200-8e16-4f384deaf236 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1297 microversion: 2.1 time: 0.105177 May 23 15:46:47.577449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 153/316] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1297 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:47.616950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7c18fb1-77b7-4dc1-88d4-c2cec8dbcc96 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "DELETE /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 204 len: 0 microversion: 2.1 time: 0.607022 May 23 15:46:47.617568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 164/317] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:46:47 2022] DELETE /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 15:46:47.626001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:47.626616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:47.634328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.634783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.635612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:47.636051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:47.836987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:47.845209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a2ef7fcc-0b20-4262-b02b-9c719fa92f7b tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1470 microversion: 2.1 time: 0.223018 May 23 15:46:47.845981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 154/318] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:47 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1470 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:48.417840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:48.417840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:48.426310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.426761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:48.427699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.428154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:48.572529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:48.580594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5a8df836-2a10-4041-a904-f3a878b63a3b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.168749 May 23 15:46:48.580749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 165/319] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:48 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:48.603562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:48.604833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:48.612251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.613167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:48.613473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.613919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:48.752331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:48.760214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24b00f92-7fd8-4078-914e-e4dd689da7ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.160721 May 23 15:46:48.760789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 155/320] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:48.862581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:48.863248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:48.865203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:48.869964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.870404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:48.871456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:48.871886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:49.004828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:49.012383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd7a3591-ab28-4f18-9a68-e83a3fc9e15d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1470 microversion: 2.1 time: 0.154227 May 23 15:46:49.013015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 166/321] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:48 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1470 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:49.597308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:49.598020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:49.604915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:49.605557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:49.606223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:49.606662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:49.785159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:49.785778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:49.802931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:49.802931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:49.802931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:49.811660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:49.911261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:49.924514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e3e625d9-a173-47c0-8482-5ddab768e6ee tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.327818 May 23 15:46:49.925914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 156/322] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:49 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:50.034135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:50.035357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:50.045816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.046363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.047493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.047994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.057492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:50.073253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-907ee68b-d58a-4dd5-b81b-8d20bab699d4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.292612 May 23 15:46:50.074175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 167/323] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:50.201336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-3cf4246f-cd97-4567-9222-e9d6ad677beb req-5fe24175-5ee0-48ca-90b5-41ac65202168 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab", "tag": "9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:50.211616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3cf4246f-cd97-4567-9222-e9d6ad677beb req-5fe24175-5ee0-48ca-90b5-41ac65202168 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.212372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3cf4246f-cd97-4567-9222-e9d6ad677beb req-5fe24175-5ee0-48ca-90b5-41ac65202168 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.226878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-3cf4246f-cd97-4567-9222-e9d6ad677beb req-5fe24175-5ee0-48ca-90b5-41ac65202168 service nova] Creating event network-changed:9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd for instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab on hv-cinder-83588 May 23 15:46:50.231955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-3cf4246f-cd97-4567-9222-e9d6ad677beb req-5fe24175-5ee0-48ca-90b5-41ac65202168 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035572 May 23 15:46:50.232830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 168/324] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:50.301697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:50.318567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61e1c67c-7a5d-4a87-9a96-5b2c8157b008 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1470 microversion: 2.1 time: 0.291026 May 23 15:46:50.320909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-23d689a2-90bb-4aa4-aaa3-f5d8d21fd641 req-235f83d0-3f01-4ee4-b3a0-4036eb0adbef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d808c02-6a3e-4e38-bfd1-986e31e8cbd2", "tag": "16595806-91fa-49b4-9688-d528f91f3cca"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:50.324649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 157/325] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:50 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1470 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:50.332780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-23d689a2-90bb-4aa4-aaa3-f5d8d21fd641 req-235f83d0-3f01-4ee4-b3a0-4036eb0adbef service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.333449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-23d689a2-90bb-4aa4-aaa3-f5d8d21fd641 req-235f83d0-3f01-4ee4-b3a0-4036eb0adbef service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.368069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-23d689a2-90bb-4aa4-aaa3-f5d8d21fd641 req-235f83d0-3f01-4ee4-b3a0-4036eb0adbef service nova] Creating event network-changed:16595806-91fa-49b4-9688-d528f91f3cca for instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 on hv-cinder-83588 May 23 15:46:50.393452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-23d689a2-90bb-4aa4-aaa3-f5d8d21fd641 req-235f83d0-3f01-4ee4-b3a0-4036eb0adbef service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087622 May 23 15:46:50.393983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 169/326] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:50.529046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:50.530103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:50.543180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.543592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.552369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.552369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.632811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'flavor' on Instance uuid 3831639c-bfc1-491d-a152-5b09cf9adf26 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:46:50.813982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:50.943056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:50.943726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:50.950288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.950734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:50.951548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:50.951983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.078325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-375f7b52-6d60-45d3-8415-5cf11fab1ebe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/action" status: 200 len: 32 microversion: 2.1 time: 0.555671 May 23 15:46:51.079786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 158/327] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:46:50 2022] POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/action => generated 32 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 15:46:51.091305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:51.092358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:51.101203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.102160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.103226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.104402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.128600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:51.138304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bff2179-7638-4862-94b5-d2101c8dae64 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.199591 May 23 15:46:51.139235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 170/328] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:50 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:51.146331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:51.146940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:51.153817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.154467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.164004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.164551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.483778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:51.488773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:51.492256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8f36d64-2527-4388-8c07-6f53c7bf45d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.406653 May 23 15:46:51.492819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 159/329] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:51 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:51.501130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:51.501130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:51.506294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f0eaff38-c470-4d31-add6-c111503787e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.364599 May 23 15:46:51.506874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 171/330] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:51.511945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.512582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.513553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.514046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.546528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:51.546979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:51.547463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lazy-loading 'fault' on Instance uuid eb4b831b-0839-44b4-809f-268bcbbf9462 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:46:51.592819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:51.601780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b8c011a-07b4-43a5-a292-7d1c450157b7 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1305 microversion: 2.1 time: 0.107981 May 23 15:46:51.602344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 160/331] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:51 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1305 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:52.156264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:52.157324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:52.163930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.164457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.165293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.165799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.303077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:52.308747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-439bda15-d3b2-43f4-b4e3-155a7851db76 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.156255 May 23 15:46:52.313199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 172/332] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:52 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:52.409775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-8999d807-39e1-42e4-a993-b9de96110a39 req-679cdf50-e806-4980-a823-0e2bf42ec66b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb4b831b-0839-44b4-809f-268bcbbf9462", "name": "network-vif-deleted", "tag": "bd02ecbd-0cff-443a-a717-6e3927629b24"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:46:52.414112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:52.419983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=14 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:52.420774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-8999d807-39e1-42e4-a993-b9de96110a39 req-679cdf50-e806-4980-a823-0e2bf42ec66b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.421230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-8999d807-39e1-42e4-a993-b9de96110a39 req-679cdf50-e806-4980-a823-0e2bf42ec66b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.432975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-8999d807-39e1-42e4-a993-b9de96110a39 req-679cdf50-e806-4980-a823-0e2bf42ec66b service nova] Creating event network-vif-deleted:bd02ecbd-0cff-443a-a717-6e3927629b24 for instance eb4b831b-0839-44b4-809f-268bcbbf9462 on hv-cinder-83588 May 23 15:46:52.437768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-8999d807-39e1-42e4-a993-b9de96110a39 req-679cdf50-e806-4980-a823-0e2bf42ec66b service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032274 May 23 15:46:52.438523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 161/333] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:46:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:46:52.510664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:52.511284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:52.517340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.521715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.521715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.521715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.525850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:52.526593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:52.534806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.535301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.536077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.536519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.729974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:52.734170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:52.738021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20eb9c43-ef82-4834-9036-42bdf8d5edd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.231354 May 23 15:46:52.738589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 173/334] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:52 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:52.743530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:52.744143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:52.744552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6d9bba73-0435-40fc-a820-a9e9a4d6a11d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.225347 May 23 15:46:52.745193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 162/335] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:52.751165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.751618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.752388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.752829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.784372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:52.784733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:52.785172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lazy-loading 'fault' on Instance uuid eb4b831b-0839-44b4-809f-268bcbbf9462 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:46:52.826585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:52.835113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bc2316dc-63dc-4557-9ac8-bae65a5614c8 tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1305 microversion: 2.1 time: 0.095557 May 23 15:46:52.835682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 174/336] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:52 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1305 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:53.325984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:53.328846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:53.333556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.334261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.334998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.335603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.474522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:53.486905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-913930b5-ecc0-45dd-92b6-79083ef5e8c5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.164629 May 23 15:46:53.487993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 163/337] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:53 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:53.755492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:53.756110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:53.761353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:53.761976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:53.762303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.762651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.763439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.763878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.768595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.769081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.769825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.770271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.960741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:53.964865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:53.968572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1ead047-9269-423d-a631-51c274bfcbb6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.216918 May 23 15:46:53.969133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 175/338] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:53 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:53.972982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cdfbd353-bcb0-41e0-8e5d-f0017578a94b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.215379 May 23 15:46:53.974320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:53.974931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:53.981050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.981494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.982264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:53.982698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:53.982875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 164/339] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:54.015033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.015517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.015970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lazy-loading 'fault' on Instance uuid eb4b831b-0839-44b4-809f-268bcbbf9462 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:46:54.058370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1687d5d160e24bbaadab47bfd30e59f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '485c8c6de3d34ae08491b570aa6c2520', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:54.066706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ec8c25b-34af-4e6f-9d34-76b9d1c66b6f tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 200 len: 1305 microversion: 2.1 time: 0.096515 May 23 15:46:54.067722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 176/340] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:53 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 1305 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:54.505279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:54.506619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:54.513238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.513750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.514523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.514954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.804357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:54.816088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52c07d37-09e1-45be-bf04-3da736af89fa tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.315181 May 23 15:46:54.816944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 165/341] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:54 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:54.986171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:54.988086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:54.990220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:54.991026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:54.994995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.995575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.997028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.997433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.997770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.997884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:54.998638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:54.999269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:55.200938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:55.207201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:55.208863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eb6c23f1-ce5b-454f-bd9a-d86a95679a7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.226798 May 23 15:46:55.210194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 177/342] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:54 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:55.214326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:55.215076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] [instance: eb4b831b-0839-44b4-809f-268bcbbf9462] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:55.220118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-43e3b40f-86a5-4f0e-95bf-d9d5b896f821 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.234316 May 23 15:46:55.221438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 166/343] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:55.221593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:55.221998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:55.222827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:55.223300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:55.246703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] HTTP exception thrown: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Request method failure captured: May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 HTTP/1.1 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6w-p0_8trsLbwRKVQbWAIrljMv6RnL4y9ejtW37mGelpkDb0v8jU9DbNfInlL5tImZMikQFQ-3RvQwYdOud2rnX28C1fBxfgpESW7fSOwjlQFH8ua8EmnYji73jyWxpxSGoBPixLPsst2QXNOz3a0GSBGiWSsx9ebERVSPxB1bT2Qs May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 485c8c6de3d34ae08491b570aa6c2520 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesNegativeTest-1711902698 May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:46:55.249269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/485c8c6de3d34ae08491b570aa6c2520"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/485c8c6de3d34ae08491b570aa6c2520"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/485c8c6de3d34ae08491b570aa6c2520"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_485c8c6de3d34ae08491b570aa6c2520", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesNegativeTest-1711902698 May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 485c8c6de3d34ae08491b570aa6c2520 May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesNegativeTest-1711902698 May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesNegativeTest-1711902698-project May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 1687d5d160e24bbaadab47bfd30e59f6 May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesNegativeTest-1711902698-project May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. May 23 15:46:55.250885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'eb4b831b-0839-44b4-809f-268bcbbf9462'} May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:46:55.251785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. May 23 15:46:55.253162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:46:55.254465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] Returning 404 to user: Instance eb4b831b-0839-44b4-809f-268bcbbf9462 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:46:55.254465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9005476a-c472-45c2-a9ae-afa72946414d tempest-VolumesNegativeTest-1711902698 tempest-VolumesNegativeTest-1711902698-project] 10.222.0.32 "GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462" status: 404 len: 111 microversion: 2.1 time: 0.040904 May 23 15:46:55.254465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 178/344] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:55 2022] GET /compute/v2.1/servers/eb4b831b-0839-44b4-809f-268bcbbf9462 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:46:55.840370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:55.841877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:55.852066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:55.860158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:55.860158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:55.860158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:55.987360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:55.996372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7df24749-a760-48cf-976e-1835e141ab52 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.164007 May 23 15:46:55.997126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 167/345] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:55 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:56.226027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:56.227838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:56.241543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:56.241543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:56.245687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:56.247980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:56.248427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:56.249018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:56.249988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:56.250307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:56.254663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:56.255475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:56.265177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:46:56.467395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:56.472739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:56.477577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-49d48d4d-e1ea-4e8b-9d9b-c2d300e2f79e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.255212 May 23 15:46:56.478564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 179/346] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:56 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:56.495014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-33bb1f76-2317-4742-92f6-1be3bbf55c2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.261357 May 23 15:46:56.497538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 168/347] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:57.016456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:57.017818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:57.034203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.034744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.035729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.047317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.493604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:57.495657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:57.496644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:57.503560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f149595f-44be-4a2d-8552-d37fab340483 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.492636 May 23 15:46:57.503702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.504034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.504140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 180/348] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:57 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:57.504796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.505259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.512610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:57.513455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:57.520999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.521553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.522401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:57.522972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:57.714450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:57.725096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af365244-ace1-45ab-a169-c7675cb5984c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.233224 May 23 15:46:57.725981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 169/349] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:57 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:57.738356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:57.746154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bad49ee8-2aec-4ed7-a591-18dbf79e762b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.236948 May 23 15:46:57.746763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 181/350] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:46:58.521139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:58.521770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:58.528429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.528877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.529641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.530076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.663204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:58.670620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da2d5035-4fac-4889-bdaa-6672e9b5fb55 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.153924 May 23 15:46:58.671212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 170/351] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:58 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:58.742319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:58.742991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:58.750777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.751308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.752580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.753014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.782391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:58.785542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:58.801926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.803701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.804557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:58.805012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:58.956571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:58.964789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a3a4d6df-beb8-43cd-9d67-f811cbaaf222 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.226393 May 23 15:46:58.969612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 182/352] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:58 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:59.002821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:59.010571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-189e10a1-7fea-40d3-ac59-5b2bcf236a46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.241252 May 23 15:46:59.011343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 171/353] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:59.687734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:59.688574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:59.696341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:59.696856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:59.697674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:59.698172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:59.833018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:46:59.841098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f364d0fa-abaa-4c70-91f9-96eb673f680d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.157452 May 23 15:46:59.841741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 183/354] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:59 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:46:59.985030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:46:59.990057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:46:59.995304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:59.995844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:46:59.996915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:46:59.997348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:00.027383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:00.028179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:00.034177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:00.034634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:00.035438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:00.035938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:00.187364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:00.195351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3fb08057-e33b-4bd2-95c9-f0ceafefc21a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.217101 May 23 15:47:00.196544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 172/355] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:46:59 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:00.213450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:00.221715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f2dafc4a-bb14-4464-b100-8ecdd6a962aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.198129 May 23 15:47:00.225199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 184/356] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:00.620013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-09393fd5-d09c-46ce-8cb5-834d345f95f7 req-84ebf33d-fb8e-49cc-8f18-ca74603f3dd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab", "name": "network-vif-plugged", "status": "completed", "tag": "9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:00.629482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-09393fd5-d09c-46ce-8cb5-834d345f95f7 req-84ebf33d-fb8e-49cc-8f18-ca74603f3dd0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:00.629998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-09393fd5-d09c-46ce-8cb5-834d345f95f7 req-84ebf33d-fb8e-49cc-8f18-ca74603f3dd0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:00.641290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-09393fd5-d09c-46ce-8cb5-834d345f95f7 req-84ebf33d-fb8e-49cc-8f18-ca74603f3dd0 service nova] Creating event network-vif-plugged:9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd for instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab on hv-cinder-83588 May 23 15:47:00.647171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-09393fd5-d09c-46ce-8cb5-834d345f95f7 req-84ebf33d-fb8e-49cc-8f18-ca74603f3dd0 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031259 May 23 15:47:00.649154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 173/357] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:00.858759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:00.859694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:00.868017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:00.868740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:00.869600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:00.870148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:01.011280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:01.022331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-86d79cac-9ccb-4e88-a4b2-58d98ad57ef1 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.168390 May 23 15:47:01.024174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 185/358] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:00 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:01.214980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:01.215748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:01.225917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:01.225917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:01.225917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:01.225917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:01.245455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:01.246398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:01.254849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:01.255333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:01.256105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:01.256533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:01.452182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:01.461870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48791f7e-2282-46b8-8fe1-22cb90564765 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.253008 May 23 15:47:01.462722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 174/359] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:01 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:01.479972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:01.487584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04984572-140c-497a-bc67-673617745ec5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.251269 May 23 15:47:01.488148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 186/360] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:02.039487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:02.045256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:02.047042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.047849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.048290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.049227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.184733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:02.192689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aaca98f0-4259-4416-9f57-b28aeff7ff09 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.157929 May 23 15:47:02.193246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 175/361] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:02 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:02.480471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:02.481260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:02.488540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.488794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.489649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.490096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.506209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:02.507036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:02.516592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.517359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.518358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:02.519006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:02.524473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:02.766013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:02.770310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:02.775642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14d32f99-14fd-4c24-a3b4-83606ff6b5fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.299955 May 23 15:47:02.776491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 187/362] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:02 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:02.778584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-675e97e1-e3cc-447d-9664-2b8377a6eb67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.277756 May 23 15:47:02.779171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 176/363] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:03.212883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:03.213911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:03.225580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.226317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:03.228403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.237291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:03.414675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:03.421279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8caf2a15-4b5b-485b-b66c-b0883b06351c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.215031 May 23 15:47:03.432098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 188/364] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:03 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:03.804047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:03.804047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:03.808756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.809230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:03.809925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:03.810244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:03.811332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.812218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:03.822060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.822810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:03.823890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:03.824780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:04.021012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:04.028663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b193a8d3-e5a8-4f8f-90b0-d2710fbaa9c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.238903 May 23 15:47:04.029234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 177/365] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:03 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:04.033618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:04.041286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fa052420-fd54-4b88-baf5-f7e12525dd94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.248725 May 23 15:47:04.041984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 189/366] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:04.437805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:04.438428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:04.445675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:04.446224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:04.447035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:04.447496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:04.587957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:04.595937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f5cb794a-9530-429a-a0c8-d750db71d95e tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.162046 May 23 15:47:04.596549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 178/367] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:04 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:04.805142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-fdbfd7e3-a805-4f74-a979-9753948e5aed req-a1b23332-2bd9-4712-8000-e86d41bc1e06 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d808c02-6a3e-4e38-bfd1-986e31e8cbd2", "name": "network-vif-plugged", "status": "completed", "tag": "16595806-91fa-49b4-9688-d528f91f3cca"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:04.814182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-fdbfd7e3-a805-4f74-a979-9753948e5aed req-a1b23332-2bd9-4712-8000-e86d41bc1e06 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:04.814783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-fdbfd7e3-a805-4f74-a979-9753948e5aed req-a1b23332-2bd9-4712-8000-e86d41bc1e06 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:04.826145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-fdbfd7e3-a805-4f74-a979-9753948e5aed req-a1b23332-2bd9-4712-8000-e86d41bc1e06 service nova] Creating event network-vif-plugged:16595806-91fa-49b4-9688-d528f91f3cca for instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 on hv-cinder-83588 May 23 15:47:04.831317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-fdbfd7e3-a805-4f74-a979-9753948e5aed req-a1b23332-2bd9-4712-8000-e86d41bc1e06 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030044 May 23 15:47:04.831887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 190/368] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:05.045677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:05.046357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:05.053025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.053820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.054600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.055226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.060751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:05.061701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:05.067901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.068349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.069300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.069911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.270163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:05.275181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:05.278211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e4b7049e-b2c8-4cab-9a1f-8c97f0f2b5b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.236278 May 23 15:47:05.278796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 179/369] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:05 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:05.284104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a8435c4-19c7-4b00-ad89-267f3888bf86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.229204 May 23 15:47:05.284713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 191/370] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:05.612389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:05.614128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:05.622104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.622484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.623341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:05.623903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:05.763829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:05.771376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-91c3c3f5-011d-4eaf-a87c-4858f9ec4959 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.162907 May 23 15:47:05.771937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 180/371] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:05 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:06.295232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:06.295838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:06.301990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.302433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.303216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.303658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.304065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:06.304684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:06.311221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.311676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.312440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.312884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.494760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:06.503767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6cfaae4c-6470-44d2-8a10-c6d81510f76f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.212421 May 23 15:47:06.508031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:06.508548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 192/372] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:06 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:06.516475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f0dfc260-0766-464a-935c-b2e1c0dd2dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.216142 May 23 15:47:06.517135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 181/373] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:06.787413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:06.788031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:06.795507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.795955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.796779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:06.797213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:06.927836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:06.935464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3882489f-fb9d-478d-b676-565a58efd1bf tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.151943 May 23 15:47:06.936023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 193/374] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:06 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:07.521176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:07.521797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:07.528437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.529242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:07.530137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.530577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:07.534328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:07.535164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:07.543045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.543728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:07.544697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.545380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:07.747227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:07.749913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:07.755538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e432c8af-65b8-4b37-9181-45f4f993380e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.238575 May 23 15:47:07.756354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 182/375] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:07 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:07.758058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0c158498-5b6f-4945-94de-20021a6d37cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.228985 May 23 15:47:07.758816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 194/376] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:07.952495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:07.953107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:07.959986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.960434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:07.970715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:07.970715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:08.092319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:08.100005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24c215db-8aad-47fd-a9ee-b1da8d122b66 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1313 microversion: 2.1 time: 0.151454 May 23 15:47:08.100562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 183/377] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:07 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1313 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:08.772935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:08.773557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:08.773817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:08.774909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:08.780217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:08.781382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:08.784683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:08.784786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:08.784891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:08.784891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:08.785035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:08.785035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:08.979331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:08.986484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:08.994445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-927bb2a9-6794-4693-a087-dbb4fc2d5412 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.224321 May 23 15:47:08.994755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bcb526cf-446b-451c-be38-0b9d575350c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.225635 May 23 15:47:08.995025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 184/378] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:09.005292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 195/379] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:08 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:09.116990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:09.117740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:09.124412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:09.124942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:09.125794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:09.126298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:09.302168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:09.309777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7ea32d2e-2c65-4739-b1a8-f2d25d6cc394 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1496 microversion: 2.1 time: 0.196899 May 23 15:47:09.312768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 185/380] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:09 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1496 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:09.747254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1709434413", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "networks": [{"uuid": "fde7f6c3-1fe4-4f6b-9aa5-195b397c0f47"}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:09.799631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] validate_networks() for [('fde7f6c3-1fe4-4f6b-9aa5-195b397c0f47', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:47:10.011064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:10.012719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:10.020266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.020721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.021496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.021935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.226686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:47:10.227410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:47:10.227799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:47:10.228207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:47:10.228780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:47:10.249871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.250332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.251216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.251827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:10.252667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.261411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-297351a4-fefa-48a9-8f32-6cca68084ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.254214 May 23 15:47:10.263971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 186/381] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:10 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:10.273249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Getting quotas for project 7e22027c2e5948a6ba154c837a7673a9. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:47:10.276339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:10.276912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:10.278361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Getting quotas for user cc4c3405e93141fc851f22e49eee09bd and project 7e22027c2e5948a6ba154c837a7673a9. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:47:10.284232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.284679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.285451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.285883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.290126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:47:10.292547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:47:10.395822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-01e61d1e-e950-4ea6-be0f-ff115fe4bcb3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.652541 May 23 15:47:10.396567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 196/382] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:47:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 654 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:47:10.406856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a54a39f-213b-40bf-8b59-7aa20a12ec17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:10.407587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a54a39f-213b-40bf-8b59-7aa20a12ec17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:10.431739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:10.440390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6cb4399b-eb51-4d02-ab83-f108546bfe88 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.167824 May 23 15:47:10.440956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 187/383] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:10.463183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a54a39f-213b-40bf-8b59-7aa20a12ec17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:10.470130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a54a39f-213b-40bf-8b59-7aa20a12ec17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1219 microversion: 2.1 time: 0.067133 May 23 15:47:10.470735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 197/384] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:10 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1219 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:10.574757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:10.575314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:10.582395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.582880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.583616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:10.590745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:10.612078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:47:11.193389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9c2a36e-cf46-4a45-88f4-01ee5cef3d62 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "DELETE /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 204 len: 0 microversion: 2.1 time: 0.622194 May 23 15:47:11.194107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 188/385] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:47:10 2022] DELETE /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 0 bytes in 624 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 15:47:11.202070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:11.202911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:11.209070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.209529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.211070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.211547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.217412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:11.278342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:11.278973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:11.285535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.285989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.286765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.287240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.382945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:11.391566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0dcf093a-f8c2-4c18-9756-fc2e9a6c518d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1502 microversion: 2.1 time: 0.192864 May 23 15:47:11.392474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 198/386] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:11 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1502 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:11.443326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:11.450897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6f8e5a5b-d769-445b-ac31-de731472caa2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.176825 May 23 15:47:11.451577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 189/387] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:11 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:11.456785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:11.457401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:11.463724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.467314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.467314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:11.467314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:11.488831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3065f5c8-4a7c-4426-9c1a-e5d800f3ad6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:11.489499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3065f5c8-4a7c-4426-9c1a-e5d800f3ad6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:11.566707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3065f5c8-4a7c-4426-9c1a-e5d800f3ad6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:11.577084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3065f5c8-4a7c-4426-9c1a-e5d800f3ad6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1219 microversion: 2.1 time: 0.093287 May 23 15:47:11.577731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 190/388] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:11 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:11.638262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:11.646152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a5c076cc-98e8-4a90-94f3-f55afaabf9f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.193102 May 23 15:47:11.646719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 199/389] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:12.409680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:12.410274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:12.416930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.417382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.418153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.418591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.476022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:12.476645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:12.483664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.493418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.493418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.493418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.591875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:12.600471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-82eedc24-4634-4a36-be54-80d6a94dbaab tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1502 microversion: 2.1 time: 0.195234 May 23 15:47:12.601089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 191/390] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:12 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1502 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:12.605551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:12.606160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:12.612495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.612993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.614039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.614481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.618902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:12.643546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:12.651388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2657d041-2a36-48bc-9aec-4b49e65bbaa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.187756 May 23 15:47:12.651967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 200/391] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:12 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:12.670874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:12.670874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:12.671186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.671656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.672415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:12.672849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:12.678018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:12.685834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-168cd454-4764-454b-8890-5b4127c6d69b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1223 microversion: 2.1 time: 0.083666 May 23 15:47:12.686423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 192/392] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:12 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1223 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:12.805281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:12.812817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-110e4fd4-cc4f-46ec-be80-17e7d0d95d9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.152362 May 23 15:47:12.813421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 201/393] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:13.617329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:13.617938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:13.625559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.626001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.627386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.627818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.668869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:13.669650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:13.676383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.676823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.677590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.678016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.822377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:13.831521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a143f72a-a6b3-4dea-860a-fa109dcfe655 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1502 microversion: 2.1 time: 0.217987 May 23 15:47:13.833543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 193/394] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:13 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1502 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:13.847716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:13.851370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:13.855458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.855905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.856839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.857071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:13.857623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.877936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4bafac7f-6e0e-4958-a54b-84ad0eddfd7d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.213008 May 23 15:47:13.888157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 202/395] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:13 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:13.893489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:13.894205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:13.900915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.901599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.902663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:13.903479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:13.959459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:13.969466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9851106b-d99e-4294-9a19-6d077dec5eb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1223 microversion: 2.1 time: 0.134668 May 23 15:47:13.971938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 194/396] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:13 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1223 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:14.055954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:14.063346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9fbeee9e-77b4-4848-bb98-39e6be1d0c9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1363 microversion: 2.1 time: 0.173784 May 23 15:47:14.063938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 203/397] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:14.850283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:14.850854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:14.853985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:14.859575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:14.860028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:14.861305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:14.861746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:14.898747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:14.900143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:14.906479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:14.907215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:14.908134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:14.908563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.441526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:15.452489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a11feeca-08b7-4fa9-8641-c088a2a8e794 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1502 microversion: 2.1 time: 0.606047 May 23 15:47:15.453685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 195/398] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:14 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1502 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:15.462597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:15.467371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:15.478276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.478888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.480058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.480679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.541038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:15.552288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-50988557-6647-4c12-9344-e2b184a968bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.661303 May 23 15:47:15.555725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 204/399] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:14 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:15.560867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:15.561621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:15.575456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:15.578569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.580604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.581884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.583703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.593177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-49345813-9690-466c-bbc7-de123ad31d7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1271 microversion: 2.1 time: 0.137711 May 23 15:47:15.593751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 196/400] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:15 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1271 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:15.816479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:15.823892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4a4a9422-3596-4aad-8c0e-bfdb233cc5ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1550 microversion: 2.1 time: 0.266844 May 23 15:47:15.824458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 205/401] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1550 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:15.841056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:15.841697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:15.848560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.849124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.850084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:15.850532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:15.966164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb56e04b-c687-43b8-bc07-73c13910e1ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.129072 May 23 15:47:15.970810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 197/402] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 15:47:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-interface => generated 285 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:16.253833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-3c767bbc-d148-4698-8aa8-96050e83696b req-1236c169-73d2-4c8f-8e61-8a654af4fe95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab", "name": "network-vif-deleted", "tag": "9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:16.270352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:16.272898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3c767bbc-d148-4698-8aa8-96050e83696b req-1236c169-73d2-4c8f-8e61-8a654af4fe95 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.274932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3c767bbc-d148-4698-8aa8-96050e83696b req-1236c169-73d2-4c8f-8e61-8a654af4fe95 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.287423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-3c767bbc-d148-4698-8aa8-96050e83696b req-1236c169-73d2-4c8f-8e61-8a654af4fe95 service nova] Creating event network-vif-deleted:9527f9e6-d3c2-4a20-b1a5-4b383c58a8fd for instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab on hv-cinder-83588 May 23 15:47:16.293290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-3c767bbc-d148-4698-8aa8-96050e83696b req-1236c169-73d2-4c8f-8e61-8a654af4fe95 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046420 May 23 15:47:16.294361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 206/403] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:16.469809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:16.470452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:16.476660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.487402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.487402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.487402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.572053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:16.572354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:16.573443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:16.583300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.583979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.584937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.585710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.610153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f97517bb-936c-426d-9e41-6e1f5f3983b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1460 microversion: 2.1 time: 0.144128 May 23 15:47:16.610770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 198/404] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:16 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1460 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:16.615297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:16.618455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:16.637940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.638394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.639202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:16.639640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:16.773096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:16.781333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0fca1e9d-7b72-4576-9d88-92590b919429 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1279 microversion: 2.1 time: 0.169482 May 23 15:47:16.784111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 199/405] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:16 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1279 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:16.950978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:16.962851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f973a414-4411-4db5-baf2-f35243dbec6b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.396044 May 23 15:47:16.963682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 207/406] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:16 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:17.629853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:17.630469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:17.643173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.645681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.646668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.647611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.698618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.699389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.700182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lazy-loading 'fault' on Instance uuid dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:47:17.757654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:17.772829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-974d3b6e-8a71-444d-93f2-a7d516d62e8f tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 200 len: 1303 microversion: 2.1 time: 0.148346 May 23 15:47:17.784691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 200/407] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:17 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 1303 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:17.804209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:17.805656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:17.813311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.814421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.815464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.816232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.891184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:17.901559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ce4ce37-8844-4c0c-b79f-69799473ca55 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1289 microversion: 2.1 time: 0.101753 May 23 15:47:17.902688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 208/408] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:17 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1289 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:17.980288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:17.980919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:17.995493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.995493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:17.995677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:17.996053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.254192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:18.267559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cbbf6573-c9d3-4bdb-8ad8-199105f337de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1501 microversion: 2.1 time: 0.291041 May 23 15:47:18.268567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 201/409] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:17 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1501 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:18.286288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0fef7576-7fc3-4a73-b6c7-ed775abdb8eb"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:18.287409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:18.294037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.294783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.296181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.296713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.316152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-af4839e8-7220-4573-8dc6-3da76198095b req-d48be22f-e5af-4b67-8474-32629f44e48e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d808c02-6a3e-4e38-bfd1-986e31e8cbd2", "tag": "16595806-91fa-49b4-9688-d528f91f3cca"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:18.326778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-af4839e8-7220-4573-8dc6-3da76198095b req-d48be22f-e5af-4b67-8474-32629f44e48e service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.328762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-af4839e8-7220-4573-8dc6-3da76198095b req-d48be22f-e5af-4b67-8474-32629f44e48e service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.328983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Cannot 'attach_volume' instance 3831639c-bfc1-491d-a152-5b09cf9adf26 while it is in vm_state rescued May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/os-volume_attachments HTTP/1.1 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 74 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:47:18.331011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {"volumeAttachment": {"volumeId": "0fef7576-7fc3-4a73-b6c7-ed775abdb8eb"}} May 23 15:47:18.341551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Cannot 'attach_volume' instance 3831639c-bfc1-491d-a152-5b09cf9adf26 while it is in vm_state rescued May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3831639c-bfc1-491d-a152-5b09cf9adf26', 'body': {'volumeAttachment': {'volumeId': '0fef7576-7fc3-4a73-b6c7-ed775abdb8eb'}}} May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 in vm_state rescued. Cannot attach_volume while the instance is in this state. May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:47:18.342992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3831639c-bfc1-491d-a152-5b09cf9adf26 while it is in vm_state rescued May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:18.346986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3831639c-bfc1-491d-a152-5b09cf9adf26 while it is in vm_state rescued May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 409 to user: Cannot 'attach_volume' instance 3831639c-bfc1-491d-a152-5b09cf9adf26 while it is in vm_state rescued {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ffa5a14d-d94b-483f-b537-0e3e8624e632 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.050744 May 23 15:47:18.349475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 209/410] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:47:18 2022] POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/os-volume_attachments => generated 152 bytes in 52 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 23 15:47:18.349987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-af4839e8-7220-4573-8dc6-3da76198095b req-d48be22f-e5af-4b67-8474-32629f44e48e service nova] Creating event network-changed:16595806-91fa-49b4-9688-d528f91f3cca for instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 on hv-cinder-83588 May 23 15:47:18.349987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-af4839e8-7220-4573-8dc6-3da76198095b req-d48be22f-e5af-4b67-8474-32629f44e48e service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035518 May 23 15:47:18.350157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:18.350249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 202/411] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:47:18.350331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:18.356093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.356550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.357349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.357781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.650181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-efa30c97-5892-4433-b166-aa007a3e16bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/action" status: 202 len: 0 microversion: 2.1 time: 0.305461 May 23 15:47:18.650944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 210/412] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:47:18 2022] POST /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26/action => generated 0 bytes in 308 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:47:18.661142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:18.662504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:18.676073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.676780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.681507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.682412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.792711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:18.793416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:18.801578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.801856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.803273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.803717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.820969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] HTTP exception thrown: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Request method failure captured: May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab HTTP/1.1 May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6xWdOJJ0acu0IAPYvp1S4SiFf302g_-ZC2qpiGWkVyg_Y3-TuTng7HpI-qQ6_ffzog7Q88STMGHBXpTfdrcv_H-Z94vBDbrkYq-6rOcoi6cWPkRh7TN2dzYP4IdV0QGnTyfSheMS8scDt-4aCKqb6Fja6M-bG-Qlz1aLpHbAlpZYSs May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: b0c39de631f44292b3b0875386f17f3f May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesActionsTest-906556723 May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:47:18.841717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_b0c39de631f44292b3b0875386f17f3f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesActionsTest-906556723 May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: b0c39de631f44292b3b0875386f17f3f May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesActionsTest-906556723 May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesActionsTest-906556723-project May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: bcaa4a7ea30242128c78eee9c2c73dfe May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesActionsTest-906556723-project May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. May 23 15:47:18.842957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab'} May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:47:18.844105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. May 23 15:47:18.845799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:47:18.847839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Returning 404 to user: Instance dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:47:18.847839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-54344987-3459-4fa7-8806-dae61fed6e29 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab" status: 404 len: 111 microversion: 2.1 time: 0.036578 May 23 15:47:18.847839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 211/413] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:18 2022] GET /compute/v2.1/servers/dd4b9c42-57ad-4d6d-89f0-f10bb7d509ab => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 15:47:18.923677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:18.924297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:18.931764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.932212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:18.932977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:18.933416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:19.370161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:19.386552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c3eaac5-353e-4a8e-ac18-31261112f096 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1509 microversion: 2.1 time: 0.730431 May 23 15:47:19.391479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 203/414] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:18 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1509 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:19.395063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1946312771", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "networks": [{"uuid": "ad45c8b8-f9f9-4607-b7a6-9f5ec3d7435f"}]}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:19.426230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:19.440279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-51c3e640-9c4b-47e6-96c1-4923d8595556 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.519839 May 23 15:47:19.451395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 212/415] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:18 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:19.462716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] validate_networks() for [('ad45c8b8-f9f9-4607-b7a6-9f5ec3d7435f', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:47:20.287648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:47:20.288801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:47:20.289663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:47:20.291548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:47:20.292185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:47:20.309788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.310603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.311635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.312277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.331252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Getting quotas for project b0c39de631f44292b3b0875386f17f3f. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:47:20.335485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Getting quotas for user bcaa4a7ea30242128c78eee9c2c73dfe and project b0c39de631f44292b3b0875386f17f3f. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:47:20.348765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:47:20.351139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:47:20.407320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:20.408587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:20.411558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:20.415671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.416124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.416886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.417315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.470599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=8 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:20.526853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-801b69f2-29e9-4c84-9288-ff7a4b17e88b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.137776 May 23 15:47:20.527780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 204/416] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:47:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 1140 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:47:20.533071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:20.534347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:20.542557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.543817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.550160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:20.551424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:20.739251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:20.748908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2438b0c4-ff44-4045-9a7b-8e3a9d72b84e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1509 microversion: 2.1 time: 0.345662 May 23 15:47:20.750065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 213/417] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:20 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1509 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:20.755131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-44d06419-7015-47cd-9b3f-8a49a1e85b97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:20.755885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-44d06419-7015-47cd-9b3f-8a49a1e85b97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:20.841114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-44d06419-7015-47cd-9b3f-8a49a1e85b97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:20.852330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-44d06419-7015-47cd-9b3f-8a49a1e85b97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1214 microversion: 2.1 time: 0.100800 May 23 15:47:20.861993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 214/418] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:20 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1214 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:20.869754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:20.877477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-85b6844c-aacf-4420-bdc6-bf114b0f90b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.348500 May 23 15:47:20.878186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 205/419] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:20 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:21.064744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-75b7e725-beee-4834-a71c-171b97fad9d1 req-0faa29d0-b4aa-403a-980f-81f880170eff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "45589704-9ca6-46fb-b1cd-8ba795032e49", "tag": "9063491e-7ccf-41db-93ec-8b36f9d72163"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:21.081258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-75b7e725-beee-4834-a71c-171b97fad9d1 req-0faa29d0-b4aa-403a-980f-81f880170eff service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:21.081769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-75b7e725-beee-4834-a71c-171b97fad9d1 req-0faa29d0-b4aa-403a-980f-81f880170eff service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:21.094880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-75b7e725-beee-4834-a71c-171b97fad9d1 req-0faa29d0-b4aa-403a-980f-81f880170eff service nova] Creating event network-changed:9063491e-7ccf-41db-93ec-8b36f9d72163 for instance 45589704-9ca6-46fb-b1cd-8ba795032e49 on hv-cinder-83588 May 23 15:47:21.100698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-75b7e725-beee-4834-a71c-171b97fad9d1 req-0faa29d0-b4aa-403a-980f-81f880170eff service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039553 May 23 15:47:21.102389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 215/420] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:21.766798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:21.769385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:21.780244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:21.780711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:21.781525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:21.781912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:21.875610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:21.877229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:21.891444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:21.892374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:21.893596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:21.894592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:21.997376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:22.007239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fba68689-c8d5-46c1-a0a0-2ba474d5cc1c tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1218 microversion: 2.1 time: 0.140174 May 23 15:47:22.007919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 216/421] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:21 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1218 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:22.026955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:22.026955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:22.031539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:22.033068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:22.034078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:22.034737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:22.045631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:22.170610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:22.180692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d54b3137-0460-4184-9e37-74a0ae21369b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1509 microversion: 2.1 time: 0.418443 May 23 15:47:22.181289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 206/422] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:21 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1509 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:22.290349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:22.299607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-040b79d7-d0bd-4e58-b8c2-47363b3ffb3a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.280021 May 23 15:47:22.299607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 217/423] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:22 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:23.026588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:23.027724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:23.042145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.043124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.044457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.045451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.199025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:23.200409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:23.208030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.208755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.209617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.210046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.447437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:23.463826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25727778-4382-4eb8-888e-ce89e5a2b409 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1210 microversion: 2.1 time: 0.443221 May 23 15:47:23.474304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 207/424] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:23 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1210 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:23.479317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:23.479839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:23.486540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.487423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.488298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:23.488731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:23.700726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:23.709047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6ffa83c2-6382-403b-a02c-900de1062759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1509 microversion: 2.1 time: 0.515152 May 23 15:47:23.715320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 218/425] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:23 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1509 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:23.783962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:23.791455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-59e99afd-4e77-4fa6-aa37-562c892b5ba1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.316009 May 23 15:47:23.815343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 208/426] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:23 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:24.481943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:24.482561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:24.491901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.492373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.493169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.493739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.638481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:24.650972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76fe6f24-216a-4350-9926-5f456a86dc7b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1274 microversion: 2.1 time: 0.172821 May 23 15:47:24.654837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 219/427] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:24 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1274 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:24.727229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:24.727926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:24.734581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.741193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.741193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.741193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.834385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:24.836662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:24.883441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.884638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.886662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:24.888464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:24.908901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:24.918721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d2530982-edf3-48c0-a96f-df572d6ec482 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1515 microversion: 2.1 time: 0.196120 May 23 15:47:24.925662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 209/428] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:24 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1515 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:25.193408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:25.205808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02a2b0bf-27c9-4779-98ea-a069e6d0e873 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.379063 May 23 15:47:25.208366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 220/429] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:24 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:25.675584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:25.676457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:25.703123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:25.709880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:25.711383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:25.712649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:25.724689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:25.910533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:25.917874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-65918d86-a42e-4b71-938c-abcf91bf7a65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1326 microversion: 2.1 time: 0.247147 May 23 15:47:25.918489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 210/430] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:25 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1326 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:26.223840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:26.224452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:26.238492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:26.239305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:26.240365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:26.240800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:26.393301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:26.400927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-695a4627-064a-4d57-84d9-330fb006a3b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.180714 May 23 15:47:26.401514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 221/431] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:26 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:26.934359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:26.934971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:26.942903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:26.943482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:26.944250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:26.944681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:27.084460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:27.092139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cabc0d5a-3e80-4b70-b55c-5033ec2797eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1326 microversion: 2.1 time: 0.161660 May 23 15:47:27.092703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 211/432] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:26 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1326 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:27.419254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:27.422173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:27.438059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:27.443893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:27.443893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:27.443893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:27.741325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:27.752411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3e0795a0-b57e-4cae-87d9-8465f6bf8dac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.337245 May 23 15:47:27.762535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 222/433] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:27 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:27.807899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1f98a843-6f6c-4587-90a7-34d8670072da tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: .version_select at 0x7f47f9a1d160>, body: {"keypair": {"name": "tempest-keypair-1932700632", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD6UJWnA6Uq0er17buFNZX8iO+mHps6FSQWrPFj0lrealeQR1dt7JFCSkiGjB2kSvBm8kIPL4Cg5ExzhF6/f/KYsg8pec2/yFlnJ41ukdCRtgd+urQYkxLztpsTCvyhkFA=="}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:27.822992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-1f98a843-6f6c-4587-90a7-34d8670072da tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:47:27.827271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-1f98a843-6f6c-4587-90a7-34d8670072da tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for user 606ff49786a04aa5b677cbb9a57b216c and project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:47:27.941030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1f98a843-6f6c-4587-90a7-34d8670072da tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.136971 May 23 15:47:27.946716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 212/434] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:47:27 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:28.111603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:28.120207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:28.137121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:28.137575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:28.138575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:28.139072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:28.360271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:28.376857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b15e4d7c-c343-4d42-be4b-e15be5ef723b tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.264291 May 23 15:47:28.376857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 223/435] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:28 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:28.535742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-441bd3c8-4850-4bfc-b972-44de0ec46a89 req-a2e773bb-83d3-4638-856f-d48d43005c31 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ccdbb78-2dea-4669-8cf0-3a841967361e", "tag": "e6c38da1-83aa-4895-9093-c406ce11e1dc"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:28.549091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-441bd3c8-4850-4bfc-b972-44de0ec46a89 req-a2e773bb-83d3-4638-856f-d48d43005c31 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:28.549546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-441bd3c8-4850-4bfc-b972-44de0ec46a89 req-a2e773bb-83d3-4638-856f-d48d43005c31 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:28.562773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-441bd3c8-4850-4bfc-b972-44de0ec46a89 req-a2e773bb-83d3-4638-856f-d48d43005c31 service nova] Creating event network-changed:e6c38da1-83aa-4895-9093-c406ce11e1dc for instance 5ccdbb78-2dea-4669-8cf0-3a841967361e on hv-cinder-83588 May 23 15:47:28.568570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-441bd3c8-4850-4bfc-b972-44de0ec46a89 req-a2e773bb-83d3-4638-856f-d48d43005c31 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037329 May 23 15:47:28.569454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 213/436] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:28.774107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:28.774783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:28.786805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:28.787657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:28.788646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:28.789095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:29.393559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:29.394281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:29.401140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:29.401478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:29.402394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:29.403494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:29.404678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:29.413048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8c53d6d7-365b-4390-bd97-238267619ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.644314 May 23 15:47:29.414157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 224/437] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:28 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:29.647612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:29.660820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc6393c2-5d28-476e-9f17-b1395b57b70d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.271017 May 23 15:47:29.673748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 214/438] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:29 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:30.430968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:30.431696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:30.443494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:30.443494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:30.443494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:30.443494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:30.680709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:30.681984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:30.684416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:30.691423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:30.697712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:30.697712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:30.697712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:30.818997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:30.829952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b4b8bfff-bbd3-46fa-900e-85038fe3737f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.402722 May 23 15:47:30.831702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 225/439] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:30 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:30.930577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:30.938578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-46e688d9-685e-48fd-b341-797be3788e65 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.262801 May 23 15:47:30.940471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 215/440] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:30 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:31.848348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:31.849178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:31.856295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:31.857770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:31.858867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:31.859548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:31.887255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:31.959492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:31.965780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:31.972914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:31.977479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:31.980113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:31.982594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:32.234840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:32.246891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e6361245-94f1-440d-aad7-1d8f06dcab7e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.402112 May 23 15:47:32.248649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 226/441] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:31 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:32.256846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-3b72ea2a-3714-4184-8fb5-245f5b76b1a3 req-9f0c9213-2812-4e38-9d9b-291002a0d219 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "45589704-9ca6-46fb-b1cd-8ba795032e49", "name": "network-vif-plugged", "status": "completed", "tag": "9063491e-7ccf-41db-93ec-8b36f9d72163"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:32.268090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3b72ea2a-3714-4184-8fb5-245f5b76b1a3 req-9f0c9213-2812-4e38-9d9b-291002a0d219 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:32.269057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-3b72ea2a-3714-4184-8fb5-245f5b76b1a3 req-9f0c9213-2812-4e38-9d9b-291002a0d219 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:32.299813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-3b72ea2a-3714-4184-8fb5-245f5b76b1a3 req-9f0c9213-2812-4e38-9d9b-291002a0d219 service nova] Creating event network-vif-plugged:9063491e-7ccf-41db-93ec-8b36f9d72163 for instance 45589704-9ca6-46fb-b1cd-8ba795032e49 on hv-cinder-83588 May 23 15:47:32.303572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:32.307776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-3b72ea2a-3714-4184-8fb5-245f5b76b1a3 req-9f0c9213-2812-4e38-9d9b-291002a0d219 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057817 May 23 15:47:32.314639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 227/442] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:32.318227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3307417a-556b-45fa-9cb2-68dba777d3eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.364346 May 23 15:47:32.318834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 216/443] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:31 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:32.406314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1165086242", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b08a16eb-8530-462b-9842-46050f408ed8"}], "security_groups": [{"name": "tempest-securitygroup--953130086"}], "key_name": "tempest-keypair-1932700632", "user_data": "}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:32.554898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] validate_networks() for [('b08a16eb-8530-462b-9842-46050f408ed8', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:47:33.008395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:47:33.009761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:47:33.010252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:47:33.010835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:47:33.011429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:47:33.022218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.026991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.026991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.026991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.048560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:47:33.054260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Getting quotas for user 606ff49786a04aa5b677cbb9a57b216c and project ad4a74e93cb54747a64778b2dbea9da8. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:47:33.080285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:47:33.083765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:47:33.267457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:33.268865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:33.286325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.287223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.288206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.289698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.294911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f2186624-b439-4e25-81b2-5b084d5e63b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.894057 May 23 15:47:33.295837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 228/444] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:47:32 2022] POST /compute/v2.1/servers => generated 408 bytes in 897 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:47:33.318008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77c142ab-73fe-46bd-a64e-f56c2f50f5c2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:33.318589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77c142ab-73fe-46bd-a64e-f56c2f50f5c2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:33.409134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77c142ab-73fe-46bd-a64e-f56c2f50f5c2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:33.425682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77c142ab-73fe-46bd-a64e-f56c2f50f5c2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1246 microversion: 2.1 time: 0.115849 May 23 15:47:33.435245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 229/445] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1246 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:33.451382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:33.460331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:33.460331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.462109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.463397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:33.464079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:33.814761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:33.827461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5c4e7396-5093-45e6-96f1-bccd7bfeb6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.566540 May 23 15:47:33.829882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 217/446] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:33 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:33.846147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:33.855332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ad72f4e-bf27-4cbf-8289-179da82fe0d4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.416811 May 23 15:47:33.856040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 230/447] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:33 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:34.443624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-86faf031-dda9-410d-b4c3-759185d90bbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:34.444239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-86faf031-dda9-410d-b4c3-759185d90bbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:34.506857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86faf031-dda9-410d-b4c3-759185d90bbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:34.509543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=13 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:34.514823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-86faf031-dda9-410d-b4c3-759185d90bbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1246 microversion: 2.1 time: 0.075363 May 23 15:47:34.515669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 218/448] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1246 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:34.846131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:34.846755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:34.855520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:34.856370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:34.857861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:34.858473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:34.874659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:34.875976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:34.885513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:34.885513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:34.886085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:34.886527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:35.108268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:35.119402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e0f1c180-88cf-474b-8964-60de7c6953b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1319 microversion: 2.1 time: 0.275853 May 23 15:47:35.120737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 231/449] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:34 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1319 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:35.133307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:35.145425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-92eda4c0-6b95-443e-abbc-7e39bc623477 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.275513 May 23 15:47:35.146292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 219/450] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:34 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:35.531566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:35.532749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:35.550562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:35.551998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:35.551998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:35.552336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:35.635963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:35.648990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cf480d76-16fa-4024-9483-f1154b878557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1242 microversion: 2.1 time: 0.121383 May 23 15:47:35.661602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 220/451] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1242 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:35.811933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c010fd09-3d61-462b-afac-b7373e286d8b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Action: 'create', calling method: .version_select at 0x7f47f934de50>, body: {"keypair": {"name": "tempest-TestShelveInstance-1375701734", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHtrCQcEOGdsRuM/mjMr8uGzCFPFTjlxvuqwmqdWkyAsjY5wbZl8lPwg2NTIPK9rIB3v5YWwvCar8r5bslE8IUZbj4L9E0s59SlbLUC/85g10TdPXpz5JqgpWH6FSzSo9A=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:35.826090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-c010fd09-3d61-462b-afac-b7373e286d8b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Getting quotas for project 652580a82e9c4f3b84eda950e5613a48. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:47:35.830521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-c010fd09-3d61-462b-afac-b7373e286d8b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Getting quotas for user e6ed5088d3494139b93a7d2af4b48a12 and project 652580a82e9c4f3b84eda950e5613a48. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:47:35.972388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c010fd09-3d61-462b-afac-b7373e286d8b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.446047 May 23 15:47:35.973769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 232/452] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:47:35 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:47:36.137137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:36.138038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:36.148087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.148087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.148087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.148087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.166314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:36.167402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:36.187164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.187366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.188111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.188551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.413433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:36.421277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:36.437552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4256d7a1-7a90-4caa-82f7-4e2b08e3deb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1508 microversion: 2.1 time: 0.293135 May 23 15:47:36.437552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 221/453] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:36 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1508 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:36.437801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56746587-bc58-436c-bd76-55ec46f64e7d tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.270936 May 23 15:47:36.442777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 233/454] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:36 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:36.450581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-489e9a60-8c28-491a-ad8b-c76eefd2b838 req-b99bd04b-c9e7-4b2e-adea-b7165ea7bb0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ccdbb78-2dea-4669-8cf0-3a841967361e", "name": "network-vif-plugged", "status": "completed", "tag": "e6c38da1-83aa-4895-9093-c406ce11e1dc"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:36.459800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-489e9a60-8c28-491a-ad8b-c76eefd2b838 req-b99bd04b-c9e7-4b2e-adea-b7165ea7bb0a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.460312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-489e9a60-8c28-491a-ad8b-c76eefd2b838 req-b99bd04b-c9e7-4b2e-adea-b7165ea7bb0a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.472065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-489e9a60-8c28-491a-ad8b-c76eefd2b838 req-b99bd04b-c9e7-4b2e-adea-b7165ea7bb0a service nova] Creating event network-vif-plugged:e6c38da1-83aa-4895-9093-c406ce11e1dc for instance 5ccdbb78-2dea-4669-8cf0-3a841967361e on hv-cinder-83588 May 23 15:47:36.477232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-489e9a60-8c28-491a-ad8b-c76eefd2b838 req-b99bd04b-c9e7-4b2e-adea-b7165ea7bb0a service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035976 May 23 15:47:36.478665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 222/455] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:36.671902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:36.671902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:36.674638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.675113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.675891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:36.678076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:36.740541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:36.748317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc39f9b7-b0ff-4ea1-bb56-8654199b4e65 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1298 microversion: 2.1 time: 0.084779 May 23 15:47:36.748897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 234/456] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1298 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:37.451789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:37.452418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:37.459196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:37.459912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:37.460891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:37.461335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:37.714353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:37.726909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c0daeec-5978-4bda-80e2-2f0d4f398f97 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.279952 May 23 15:47:37.727534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 223/457] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:37 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:37.766206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:37.767615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:37.773719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:37.774140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:37.774901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:37.775358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:37.860251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:37.870217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-87f40a22-2576-41e0-b470-504882811ae1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1306 microversion: 2.1 time: 0.107954 May 23 15:47:37.873616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 235/458] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1306 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:38.744186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:38.744823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:38.752817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:38.753561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:38.754708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:38.755352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:38.888937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:38.890271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:38.905149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:38.906225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:38.907172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:38.908230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:38.995226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:38.999883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:39.010931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c15c9c51-3465-4244-a994-40a34ed43129 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1314 microversion: 2.1 time: 0.270524 May 23 15:47:39.012323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8616e754-f031-4f1c-854f-2dc3f6f3e533 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1304 microversion: 2.1 time: 0.127114 May 23 15:47:39.013182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 236/459] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1304 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:39.022353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 224/460] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:38 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1314 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:40.039553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:40.042374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:40.047861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:40.052944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:40.053719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:40.054788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:40.055473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:40.059043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:40.061886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:40.068168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:40.068984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:40.075739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:40.075739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:40.313567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:40.317343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:40.322011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f6fe580f-d89d-41a6-a0e8-ab8e5a5eebe5 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1497 microversion: 2.1 time: 0.287173 May 23 15:47:40.326057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8407956-bada-442a-8c90-aa31de4904e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.282611 May 23 15:47:40.335864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 225/461] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:40.336045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 237/462] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:40 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1497 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:40.645165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-bde51229-da6f-476d-9600-434eb23f7f7b req-41afbd13-f8d1-4c6d-9356-e91e77eae17f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:40.662571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-bde51229-da6f-476d-9600-434eb23f7f7b req-41afbd13-f8d1-4c6d-9356-e91e77eae17f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:40.664491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-bde51229-da6f-476d-9600-434eb23f7f7b req-41afbd13-f8d1-4c6d-9356-e91e77eae17f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:40.674928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-bde51229-da6f-476d-9600-434eb23f7f7b req-41afbd13-f8d1-4c6d-9356-e91e77eae17f service nova] Creating event network-changed:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 15:47:40.684233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-bde51229-da6f-476d-9600-434eb23f7f7b req-41afbd13-f8d1-4c6d-9356-e91e77eae17f service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043679 May 23 15:47:40.684839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 226/463] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:41.346183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:41.346987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:41.355604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.356064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:41.356970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.357405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:41.469317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:41.471116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:41.479139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.479859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:41.480705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.481208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:41.544563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:47:41.628151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:41.640726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04c8c827-a800-49c7-a62a-7313f933cfd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.300798 May 23 15:47:41.641380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 238/464] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:41.919279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf7e46f6-c20e-4b28-8a21-d6a2a29255dd tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "DELETE /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 204 len: 0 microversion: 2.1 time: 0.455289 May 23 15:47:41.919788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 227/465] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:47:41 2022] DELETE /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 15:47:41.927988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:41.928598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:41.934558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.935003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:41.935791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:41.936226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:42.081255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:42.089305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78374ca8-481d-47bb-add5-bd71a99af1eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1503 microversion: 2.1 time: 0.165204 May 23 15:47:42.089887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 239/466] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:41 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1503 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:42.659722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:42.660436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:42.664498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:42.668117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:42.668611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:42.673330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:42.673330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:42.809114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:42.817420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f808841e-6c9c-4757-99eb-c7d9185014ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.162659 May 23 15:47:42.818321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 228/467] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:43.109910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:43.110484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:43.112494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:43.117462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:43.117909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:43.118677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:43.119129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:43.271019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:43.279508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-404c6833-eb04-48c6-8055-e38b164a5b15 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1503 microversion: 2.1 time: 0.173337 May 23 15:47:43.280087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 240/468] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:43 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1503 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:43.833875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:43.834609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:43.841044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:43.841495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:43.842290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:43.842725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:43.845787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:43.975987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:43.984202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ff38dde6-8a6e-48d8-b6d9-d2379492bca3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.154136 May 23 15:47:43.984779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 229/469] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:44.297929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:44.298560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:44.300506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:44.305605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:44.306046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:44.306802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:44.307261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:44.682496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:44.690896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-362487de-2664-4d46-b8c7-5b587843f8e6 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1503 microversion: 2.1 time: 0.396749 May 23 15:47:44.691519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 241/470] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:44 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1503 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:45.001673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:45.008331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:45.008908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:45.009358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:45.010121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:45.010556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:45.037422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:45.149268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:45.156779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8da4e774-ab9d-4a04-9191-21e0bb4f4399 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.158895 May 23 15:47:45.157888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 230/471] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:45.712459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:45.713074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:45.719572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:45.720629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:45.721398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:45.721834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:45.839895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=12 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:45.860591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:45.868549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-917233e6-e739-4feb-ae80-2879905492b3 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1503 microversion: 2.1 time: 0.159844 May 23 15:47:45.869210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 242/472] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:45 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:46.173795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:46.174413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:46.181446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:46.181903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:46.182683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:46.183144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:46.325603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:46.334310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4fbce8fb-66c7-4654-96eb-e5d02eb6812d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.164152 May 23 15:47:46.335363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 231/473] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:46.887286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:46.887891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:46.894690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:46.895239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:46.895995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:46.896424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:47.033868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:47.042571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-772e4d3f-7f62-473a-8b6e-d83583e87bb4 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1503 microversion: 2.1 time: 0.159339 May 23 15:47:47.043188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 243/474] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:46 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:47.355013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:47.358459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:47.366454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:47.366894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:47.368009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:47.368480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:47.529565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:47.539262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f467fa26-fe61-49d5-83d0-d8c40ffe28e1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.190360 May 23 15:47:47.539848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 232/475] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:47:48.061661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:48.062304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:48.073790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.074262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.075074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.075514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.107827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-a79422b9-7b35-4f2b-babc-40447f1b571c req-d085e60b-5663-4dbe-b5f7-36c1efe0ec36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ccdbb78-2dea-4669-8cf0-3a841967361e", "name": "network-vif-deleted", "tag": "e6c38da1-83aa-4895-9093-c406ce11e1dc"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:48.116558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-a79422b9-7b35-4f2b-babc-40447f1b571c req-d085e60b-5663-4dbe-b5f7-36c1efe0ec36 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.117551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-a79422b9-7b35-4f2b-babc-40447f1b571c req-d085e60b-5663-4dbe-b5f7-36c1efe0ec36 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.133669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-a79422b9-7b35-4f2b-babc-40447f1b571c req-d085e60b-5663-4dbe-b5f7-36c1efe0ec36 service nova] Creating event network-vif-deleted:e6c38da1-83aa-4895-9093-c406ce11e1dc for instance 5ccdbb78-2dea-4669-8cf0-3a841967361e on hv-cinder-83588 May 23 15:47:48.138464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-a79422b9-7b35-4f2b-babc-40447f1b571c req-d085e60b-5663-4dbe-b5f7-36c1efe0ec36 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034561 May 23 15:47:48.139064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 233/476] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:47:48.143013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:48.152829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1217eedf-f308-42a4-9a41-7f2e073779eb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1461 microversion: 2.1 time: 0.095029 May 23 15:47:48.153409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 244/477] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:48 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1461 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:48.561519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:48.562213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:48.569295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.569744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.570521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.570952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.707851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:48.715628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d64cd3fe-3e67-49b3-a732-34e5f0343a4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.158816 May 23 15:47:48.716201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 234/478] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:48.897889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-54325b1f-b332-4aa1-91a1-9fc067316a1f req-b4cf006f-5580-4227-8044-fe65ec29fd53 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "name": "network-vif-plugged", "status": "completed", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:48.906937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-54325b1f-b332-4aa1-91a1-9fc067316a1f req-b4cf006f-5580-4227-8044-fe65ec29fd53 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:48.907416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-54325b1f-b332-4aa1-91a1-9fc067316a1f req-b4cf006f-5580-4227-8044-fe65ec29fd53 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:48.918923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-54325b1f-b332-4aa1-91a1-9fc067316a1f req-b4cf006f-5580-4227-8044-fe65ec29fd53 service nova] Creating event network-vif-plugged:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 15:47:48.924684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-54325b1f-b332-4aa1-91a1-9fc067316a1f req-b4cf006f-5580-4227-8044-fe65ec29fd53 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030724 May 23 15:47:48.925286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 245/479] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:49.170251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:49.170816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:49.178527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:49.178988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:49.179815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:49.180260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:49.213673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:49.214126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:49.214585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lazy-loading 'fault' on Instance uuid 5ccdbb78-2dea-4669-8cf0-3a841967361e {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:47:49.258539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcaa4a7ea30242128c78eee9c2c73dfe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b0c39de631f44292b3b0875386f17f3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:49.266983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8502da4a-d5c2-4685-ace4-91e712970007 tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 200 len: 1304 microversion: 2.1 time: 0.100911 May 23 15:47:49.267644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 235/480] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:49 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 1304 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:49.732651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:49.733356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:49.740150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:49.740602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:49.741362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:49.741800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:49.876087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:49.884016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4837b798-595e-4232-95bc-db81dd9f2c90 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.155587 May 23 15:47:49.884598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 246/481] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:50.284794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:50.285335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] [instance: 5ccdbb78-2dea-4669-8cf0-3a841967361e] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:50.291917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:50.292374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:50.293136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:50.293660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:50.308110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] HTTP exception thrown: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Request method failure captured: May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e HTTP/1.1 May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xWdOJJ0acu0IAPYvp1S4SiFf302g_-ZC2qpiGWkVyg_Y3-TuTng7HpI-qQ6_ffzog7Q88STMGHBXpTfdrcv_H-Z94vBDbrkYq-6rOcoi6cWPkRh7TN2dzYP4IdV0QGnTyfSheMS8scDt-4aCKqb6Fja6M-bG-Qlz1aLpHbAlpZYSs May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: b0c39de631f44292b3b0875386f17f3f May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesActionsTest-906556723 May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:47:50.309798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/b0c39de631f44292b3b0875386f17f3f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_b0c39de631f44292b3b0875386f17f3f", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesActionsTest-906556723 May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: b0c39de631f44292b3b0875386f17f3f May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesActionsTest-906556723 May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesActionsTest-906556723-project May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: bcaa4a7ea30242128c78eee9c2c73dfe May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesActionsTest-906556723-project May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. May 23 15:47:50.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '5ccdbb78-2dea-4669-8cf0-3a841967361e'} May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:47:50.313941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. May 23 15:47:50.315686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:47:50.318092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] Returning 404 to user: Instance 5ccdbb78-2dea-4669-8cf0-3a841967361e could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:47:50.318092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dbd3c3c7-c4ee-43ba-9a76-e73c5681bbfb tempest-VolumesActionsTest-906556723 tempest-VolumesActionsTest-906556723-project] 10.222.0.32 "GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e" status: 404 len: 111 microversion: 2.1 time: 0.030947 May 23 15:47:50.318092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 236/482] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:50 2022] GET /compute/v2.1/servers/5ccdbb78-2dea-4669-8cf0-3a841967361e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 15:47:50.901107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:50.901673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:50.907692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:50.908142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:50.909040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:50.909480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:51.074717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:51.082522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34b32e6c-17b1-4cd1-ac08-4a640dd7be60 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.185317 May 23 15:47:51.091281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 247/483] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:52.103455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:52.104158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:52.110801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:52.111300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:52.112122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:52.112562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:52.245607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:52.254317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da6d36e5-30d0-4e7c-af39-8261ac79598a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1371 microversion: 2.1 time: 0.154616 May 23 15:47:52.254884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 237/484] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:53.271851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:53.272655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:53.280557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:53.281042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:53.281990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:53.282426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:53.498348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:47:53.507241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6d815bf-72a0-4b13-9cbe-e9c377beed9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1566 microversion: 2.1 time: 0.239697 May 23 15:47:53.507825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 248/485] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:47:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1566 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:47:53.524031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:53.524748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:53.532351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:53.532842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:53.533617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:53.534053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:53.662074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48bbac71-c4b8-4546-873d-e81db4787f59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.141668 May 23 15:47:53.663168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 238/486] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 15:47:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-interface => generated 285 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:54.864228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-f84ae4f3-d645-4cfc-867f-d30bfb61c1d9 req-77f23651-b61b-4d42-b29a-d9285377a1bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:47:54.872512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f84ae4f3-d645-4cfc-867f-d30bfb61c1d9 req-77f23651-b61b-4d42-b29a-d9285377a1bf service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:54.872944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f84ae4f3-d645-4cfc-867f-d30bfb61c1d9 req-77f23651-b61b-4d42-b29a-d9285377a1bf service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:54.883911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-f84ae4f3-d645-4cfc-867f-d30bfb61c1d9 req-77f23651-b61b-4d42-b29a-d9285377a1bf service nova] Creating event network-changed:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 15:47:54.889154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-f84ae4f3-d645-4cfc-867f-d30bfb61c1d9 req-77f23651-b61b-4d42-b29a-d9285377a1bf service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031665 May 23 15:47:54.889740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 249/487] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:47:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:47:58.691134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:47:58.691736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:47:58.697550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:58.697999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:58.698764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:47:58.699240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:47:58.721508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5156c9ab-3a29-4b1f-9a71-20108dce38ee tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/ips" status: 200 len: 151 microversion: 2.1 time: 0.035161 May 23 15:47:58.722581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 239/488] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:47:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/ips => generated 151 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:05.640463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=17 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:05.663457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=68 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:05.663457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e1263857-63cf-45c6-8a18-ae4614c86f1d"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:05.663457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:05.663457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:05.663942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:05.663942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:05.663942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:06.420295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-939fadc3-fd37-4e50-9e2f-0bc7f1525632 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.780699 May 23 15:48:06.425970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 250/489] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:48:05 2022] POST /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 194 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:10.023287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:10.024630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:10.031529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=25 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:10.033934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=55 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:10.038024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:10.038483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:10.039317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:10.041284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:10.072022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9f17eec-fb96-40a3-b186-70f6425563a9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/ips" status: 200 len: 150 microversion: 2.20 time: 0.054152 May 23 15:48:10.072622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 240/490] 10.222.0.32 () {60 vars in 1288 bytes} [Mon May 23 15:48:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/ips => generated 150 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:48:14.690024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:14.690914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:14.710345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:14.715150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:14.716060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:14.716593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:14.741722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lazy-loading 'flavor' on Instance uuid 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:48:15.451943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-39f8ab87-f8cb-4ece-b577-9008c221ed05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action" status: 202 len: 0 microversion: 2.20 time: 0.770102 May 23 15:48:15.456793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 251/491] 10.222.0.32 () {62 vars in 1318 bytes} [Mon May 23 15:48:14 2022] POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action => generated 0 bytes in 775 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 15:48:15.462985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:15.463646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:15.481653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:15.482495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:15.484332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:15.484980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:15.660397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:15.669094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:15.671672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:15.673155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25c3f231-7d30-4f80-aa94-f55e15221346 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.215691 May 23 15:48:15.673938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 241/492] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:16.241329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-84497ecc-559f-488d-96db-a0020b22d055 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: .version_select at 0x7f47f934d040>, body: {"keypair": {"name": "tempest-keypair-1190510603", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKs/4aryrSExm0CH+HQSjYSZ3RYSzJhpbGLhhaiu2TnTwnrpniW7xmWPDMPUF/+QU4Kh+Sn9aqtzV1sJfVtSMWHRh2bcQowzAWfGTmRI0DjFuky+SDSoDzKo2tMamjlpGg=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:16.257298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-84497ecc-559f-488d-96db-a0020b22d055 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:48:16.261452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-84497ecc-559f-488d-96db-a0020b22d055 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:48:16.347445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-84497ecc-559f-488d-96db-a0020b22d055 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.120829 May 23 15:48:16.353516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 252/493] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:48:16 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:48:16.693756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:16.695026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:16.702453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:16.702902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:16.703721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:16.704153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:16.874808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:16.891388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:16.893806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:16.895658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cad51f0d-435d-47b7-8781-cc6333d4ee8a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.206962 May 23 15:48:16.896451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 242/494] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:17.916305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:17.917710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:17.926924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:17.927385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:17.928177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:17.928788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:18.096561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:18.109191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:18.115630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:18.117030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32285be1-323f-4229-bc77-2a3e0a3af28c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.204984 May 23 15:48:18.120129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 253/495] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:17 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:19.029489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1159210315", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7e7a8587-a97e-4c11-9c4a-682adeb9c236"}], "security_groups": [{"name": "tempest-securitygroup--2032319445"}], "key_name": "tempest-keypair-1190510603", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:19.139130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:19.139576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] validate_networks() for [('7e7a8587-a97e-4c11-9c4a-682adeb9c236', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:48:19.139881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:19.147273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:19.147964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:19.152624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:19.153258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:19.368325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:19.376703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:19.378575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:19.381049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0c4002dc-06ab-4a80-bfdc-9aa67e5270e9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.246792 May 23 15:48:19.381812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 254/496] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:19.680729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:48:19.681669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:48:19.682452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:48:19.683026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:48:19.683619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:48:19.697631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:19.698079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:19.698874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:19.699348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:19.720387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:48:19.726369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:48:19.737627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:48:19.740350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:48:19.893952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d131ee4d-ea01-4e73-add1-f9e27564e0a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.867831 May 23 15:48:19.896185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 243/497] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:48:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 872 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:48:19.907747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-19a1f915-aebf-4e5c-97a9-673030fddacc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:19.908650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-19a1f915-aebf-4e5c-97a9-673030fddacc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:19.985023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-19a1f915-aebf-4e5c-97a9-673030fddacc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:19.992473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-19a1f915-aebf-4e5c-97a9-673030fddacc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1242 microversion: 2.1 time: 0.088561 May 23 15:48:19.993227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 255/498] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:19 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1242 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:20.400637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:20.401373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:20.408057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:20.408663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:20.409545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:20.410080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:20.599979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:20.609253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:20.611687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:20.612115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-42099dd0-02e4-4ff0-96c7-95e3487cbf11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.216192 May 23 15:48:20.623249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 244/499] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:21.009794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:21.011449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:21.023215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:21.023762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:21.024814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:21.025323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:21.097503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:21.106636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dca0e198-26bd-4957-85b3-e802950f45e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1246 microversion: 2.1 time: 0.100439 May 23 15:48:21.107269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 256/500] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:21 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1246 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:21.632510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:21.633510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:21.640483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:21.641041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:21.641753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:21.642474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:21.835143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:21.845815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:21.847484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:21.849225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0199796-542c-4884-bf1e-d7791871edea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.220819 May 23 15:48:21.850203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 245/501] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:22.128641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:22.130227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:22.137259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:22.138258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:22.139238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:22.140014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:22.236420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:22.247954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9ff1ce4-76bb-4822-86b9-ffea47cab84b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1302 microversion: 2.1 time: 0.127149 May 23 15:48:22.250655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 257/502] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:22 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1302 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:22.867529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:22.868288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:22.875019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:22.881498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:22.882914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:22.883683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:23.140215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:23.149444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:23.151389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:23.152832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-741d9b7d-55ad-44d0-a0a8-0529c17077ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.289138 May 23 15:48:23.153558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 246/503] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:23.270255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:23.271140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:23.281207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:23.281673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:23.284216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:23.284671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:23.552838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:23.561497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d43421b3-cde2-412b-a4b6-7251432ce295 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.296773 May 23 15:48:23.562082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 258/504] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:23 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:24.173392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:24.175196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:24.184747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:24.185621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:24.186825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:24.187655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:24.314290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-10197fd5-8f28-4650-9d2a-22efddacff46 req-4be6c613-5613-46c9-9c21-7b020a74388a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7915105-453d-453a-ad3c-656139c27d5a", "tag": "5f5a9b4b-5b24-4108-88fc-c41edee8bf44"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:24.328833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-10197fd5-8f28-4650-9d2a-22efddacff46 req-4be6c613-5613-46c9-9c21-7b020a74388a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:24.329636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-10197fd5-8f28-4650-9d2a-22efddacff46 req-4be6c613-5613-46c9-9c21-7b020a74388a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:24.358339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-10197fd5-8f28-4650-9d2a-22efddacff46 req-4be6c613-5613-46c9-9c21-7b020a74388a service nova] Creating event network-changed:5f5a9b4b-5b24-4108-88fc-c41edee8bf44 for instance c7915105-453d-453a-ad3c-656139c27d5a on hv-cinder-83588 May 23 15:48:24.369104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-10197fd5-8f28-4650-9d2a-22efddacff46 req-4be6c613-5613-46c9-9c21-7b020a74388a service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057502 May 23 15:48:24.371461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 259/505] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:48:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:48:24.420266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:24.432152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:24.433659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:24.434839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-76d829bc-8edc-4b79-b336-008db6bf7c99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.266492 May 23 15:48:24.446207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 247/506] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:24.583567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:24.584541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:24.594471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:24.594929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:24.595740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:24.596305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:24.787256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:24.795789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0ee203da-b12c-4ff1-90a1-58af84f26934 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.217261 May 23 15:48:24.796674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 260/507] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:24 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:25.453889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:25.454986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:25.460560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:25.461005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:25.461767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:25.462196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:25.604801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:25.612970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:25.614477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:25.615669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a6b4158c-295b-4ee3-9d5e-601800094137 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.166031 May 23 15:48:25.616219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 248/508] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:25.813233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:25.813767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:25.820255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:25.820697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:25.821441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:25.821866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:25.948108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:25.955371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47bf5561-f43c-48e9-aac7-d65bfe3b6ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.146029 May 23 15:48:25.955920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 261/509] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:25 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:26.187861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "784d6bce-60ef-48f4-a3bd-817c1680b9ac", "device": "/dev/sdb"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:26.189514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:26.196004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.196455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:26.197208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.197643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:26.634128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:26.634774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:26.640848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.641302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:26.642070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.642509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:26.772949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:26.780804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:26.782301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:26.783484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a43f6250-6952-44d8-b934-6a4b8c5c3f45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.153215 May 23 15:48:26.784243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 262/510] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:26.971869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:26.972476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:26.980089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.980553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:26.981302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:26.981735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:27.118428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:27.126338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d2dbc2a-a42a-4dfb-bd15-8ff81fff6901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.158261 May 23 15:48:27.126964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 263/511] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:26 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:48:27.802269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:27.802878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:27.808898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:27.809371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:27.810118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:27.810549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:27.939188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:27.947810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:27.949320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:27.950615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b1a8fb4d-6194-40be-82d4-5ac715a546d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.152733 May 23 15:48:27.951481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 264/512] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:28.136102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-2042933341", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1375701734", "security_groups": [{"name": "tempest-secgroup-smoke-1132971788"}], "networks": [{"uuid": "0d549f88-6b84-4ce1-82ad-3c3d9b4b0ea0"}], "block_device_mapping_v2": [{"uuid": "c7bcf309-96ca-4b05-9407-08949577e196", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:28.251048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] validate_networks() for [('0d549f88-6b84-4ce1-82ad-3c3d9b4b0ea0', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:48:28.619649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:48:28.620073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:48:28.621935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:48:28.622383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:48:28.623121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:48:28.632305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:28.632754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:28.633595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:28.634041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:28.651275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Getting quotas for project 652580a82e9c4f3b84eda950e5613a48. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:48:28.655458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Getting quotas for user e6ed5088d3494139b93a7d2af4b48a12 and project 652580a82e9c4f3b84eda950e5613a48. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:48:28.712990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:48:28.715200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c7bcf309-96ca-4b05-9407-08949577e196',volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:48:29.138263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-86fc30cf-d863-48af-a8e0-f5b534af03cc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.006000 May 23 15:48:29.139317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 265/513] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:48:28 2022] POST /compute/v2.1/servers => generated 406 bytes in 1008 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 23 15:48:29.145029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:29.145728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:29.153995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:29.154454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:29.155258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:29.155746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:29.174756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:29.298524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:29.306743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0be549b-7a61-4db7-b667-7cbade3fea08 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.166358 May 23 15:48:29.307449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 266/514] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:29 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:29.312064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:29.312760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:29.319638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:29.320172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:29.320999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:29.321571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:29.465712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:29.474061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:29.475760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:29.476929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-18412f08-706a-4b71-846a-0185a07a5af8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.168462 May 23 15:48:29.477488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 267/515] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:29.482306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2369e018-f3a7-4ff7-9154-6df91efc26cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:29.482918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2369e018-f3a7-4ff7-9154-6df91efc26cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:29.538473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2369e018-f3a7-4ff7-9154-6df91efc26cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:29.545574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2369e018-f3a7-4ff7-9154-6df91efc26cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1089 microversion: 2.1 time: 0.066789 May 23 15:48:29.546142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 268/516] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1089 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:29.708843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:29.732631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=6 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:29.951921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe3c8b40-0d38-4b3a-a5e9-451be608017c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "POST /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.767660 May 23 15:48:29.953166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 249/517] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:48:26 2022] POST /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49/os-volume_attachments => generated 194 bytes in 3770 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:30.323827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:30.324437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:30.331621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.332580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.333331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.333760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.486699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:30.496347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:30.496556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-990559ae-d039-4f51-aa7f-211f7b3131ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.176494 May 23 15:48:30.499134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:30.499301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 269/518] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:30 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:30.504722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.505301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.506079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.506535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.562437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:30.563050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:30.569678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.570126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.570894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:30.571360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:30.641226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:30.651838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ab95c1e7-5f06-4571-abdc-f0d1f50b842f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1139 microversion: 2.1 time: 0.093197 May 23 15:48:30.652683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 270/519] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1139 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:48:30.679417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:30.687495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:30.688994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:30.690165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2540e3c7-5bf7-42ab-b4dd-3c2e8946a9a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.198356 May 23 15:48:30.690721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 250/520] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:31.514074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:31.514728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:31.522222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.522701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.523837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.524313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.668260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:31.668883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:31.673344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:31.685159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c45a6c43-3a13-47f5-80bb-8f94b43b1e24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.175159 May 23 15:48:31.686153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 271/521] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:31 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:31.688172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.688816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.689831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.690269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.710197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:31.710812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:31.725009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.725462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.726233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:31.726670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:31.875892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:31.888640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9056a9a7-b62d-4468-bad5-ac6b76d9e94c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1195 microversion: 2.1 time: 0.223979 May 23 15:48:31.891671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 251/522] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1195 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:31.985298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:32.002512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:32.022422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:32.026188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0494beb2-bc79-422a-8ad6-e44ce6366695 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.320590 May 23 15:48:32.026695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 272/523] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:32.703981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:32.704905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:32.712622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:32.713169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:32.714014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:32.714484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:32.855630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:32.863706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-04e39e7e-8116-4146-b756-9016353dcdac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.163422 May 23 15:48:32.864288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 252/524] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:32 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:32.905521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:32.906678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:32.913814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:32.914492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:32.915318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:32.915762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:33.050559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:33.051716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:33.057799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:33.058244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:33.059035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:33.059506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:33.173792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:33.188809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e1bbcd8-0dd5-441c-98d9-dc4f32c06a12 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.286517 May 23 15:48:33.190512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 273/525] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:33.368550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-80364a2a-b2fc-40ff-8aba-6178c6a4eb07 req-aae98f6f-067c-4b96-93c0-1424492d24e0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7915105-453d-453a-ad3c-656139c27d5a", "name": "network-vif-plugged", "status": "completed", "tag": "5f5a9b4b-5b24-4108-88fc-c41edee8bf44"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:33.378661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-80364a2a-b2fc-40ff-8aba-6178c6a4eb07 req-aae98f6f-067c-4b96-93c0-1424492d24e0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:33.381901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-80364a2a-b2fc-40ff-8aba-6178c6a4eb07 req-aae98f6f-067c-4b96-93c0-1424492d24e0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:33.393069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-80364a2a-b2fc-40ff-8aba-6178c6a4eb07 req-aae98f6f-067c-4b96-93c0-1424492d24e0 service nova] Creating event network-vif-plugged:5f5a9b4b-5b24-4108-88fc-c41edee8bf44 for instance c7915105-453d-453a-ad3c-656139c27d5a on hv-cinder-83588 May 23 15:48:33.399161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-80364a2a-b2fc-40ff-8aba-6178c6a4eb07 req-aae98f6f-067c-4b96-93c0-1424492d24e0 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034221 May 23 15:48:33.400388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 274/526] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:48:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:48:33.581373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:33.589671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:33.592069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:33.593259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-faffdddd-8180-4b09-a93a-8a66ef53bf93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.546564 May 23 15:48:33.594329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 253/527] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:33.882876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:33.883505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:33.893450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:33.895547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:33.896332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:33.896933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.113569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:34.120905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8126948b-6dad-4f1b-be79-970639410bd7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.242804 May 23 15:48:34.121546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 275/528] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:33 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:48:34.213630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:34.218512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:34.226427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:34.227268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.228730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:34.229739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.296187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-70c13de4-d7aa-4ea4-8330-a5900b0110d6 req-ab9dec19-aa77-4dc8-be5b-15c3b6913f26 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:34.306437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-70c13de4-d7aa-4ea4-8330-a5900b0110d6 req-ab9dec19-aa77-4dc8-be5b-15c3b6913f26 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:34.307116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-70c13de4-d7aa-4ea4-8330-a5900b0110d6 req-ab9dec19-aa77-4dc8-be5b-15c3b6913f26 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.319224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-70c13de4-d7aa-4ea4-8330-a5900b0110d6 req-ab9dec19-aa77-4dc8-be5b-15c3b6913f26 service nova] Creating event network-changed:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 15:48:34.324244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-70c13de4-d7aa-4ea4-8330-a5900b0110d6 req-ab9dec19-aa77-4dc8-be5b-15c3b6913f26 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033128 May 23 15:48:34.325090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 276/529] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:48:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:34.423160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:34.432239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-997ef3c4-a8c6-4d3d-9fc5-b47cfec1bbfc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.227869 May 23 15:48:34.433259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 254/530] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:34.612993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:34.614098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:34.620334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:34.620784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.621535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:34.621967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:34.762179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:34.770883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:34.772469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:34.773682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ec018fd-ef17-486c-9ac4-406200e66e13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.165050 May 23 15:48:34.774233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 277/531] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:35.137750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:35.138361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:35.144427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.144873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.145656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.146086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.281472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:35.289606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99a65d58-b611-4d82-b8d6-c01c07cde9de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.155973 May 23 15:48:35.290272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 255/532] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:35 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:35.449603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:35.450223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:35.457650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.458122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.458915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.459386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.591942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:35.600026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e7c113b-a993-41e2-83f4-1edce6212204 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.154241 May 23 15:48:35.600589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 278/533] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:35.791755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:35.792397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:35.800091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.800540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.801384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:35.801848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:35.806303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:35.942349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:35.950180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:35.951738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:35.952906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae6bffd3-c3e1-4b7c-842f-b0ad3bb5f6c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1721 microversion: 2.20 time: 0.165054 May 23 15:48:35.953460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 256/534] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1721 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:36.306802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:36.307435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:36.309463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:36.314216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.314754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:36.315657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.316090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:36.457485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:36.465774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dc539d11-b9e1-4512-982d-db832b1a50fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.162897 May 23 15:48:36.466353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 279/535] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:36 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:36.619566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:36.619566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:36.624219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.625311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:36.626936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.627401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:36.774136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:36.782135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8056507f-a86e-49ae-b05e-5f81ab4d9340 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.169608 May 23 15:48:36.782829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 257/536] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:36.972450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:36.973745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:36.979821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.980269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:36.981029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:36.981463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:37.109044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:37.117173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:37.118665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:37.120003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e0d0ec8-8157-4802-8a9c-ed9101a2d8f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.152003 May 23 15:48:37.120553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 280/537] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:48:37.484341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:37.485007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:37.491112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:37.491523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:37.492280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:37.492711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:37.638167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:37.639370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5f311474-2884-451b-a590-f9cefb4a34d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1368 microversion: 2.1 time: 0.159503 May 23 15:48:37.639951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 258/538] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:37 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1368 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:37.801474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:37.802037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:37.809083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:37.809539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:37.810319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:37.810753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:37.953078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:37.960753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5fa5d87-0246-490b-9134-be0243b5bdfa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.163836 May 23 15:48:37.961318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 281/539] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:48:38.138471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:38.139003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:38.145183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.145632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.146393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.146826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.284746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:38.293374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:38.294911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:38.296110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b57962cd-153a-4d69-8d35-985e8401bad5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.161714 May 23 15:48:38.296661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 259/540] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:38.655813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:38.656421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:38.658249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:38.664068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.664532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.665325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.665768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.804313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:38.813033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-13178ea1-30c6-4e52-937f-9f7a00bfb91d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1558 microversion: 2.1 time: 0.160970 May 23 15:48:38.813611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 282/541] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:38 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:38.829476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:38.829979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:38.837072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.837533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.838291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.838727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.919235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c2add35-c5ea-4d87-99ef-3f0486ef47de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093599 May 23 15:48:38.920170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 260/542] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 15:48:38 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:38.977728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:38.978389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:38.984488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.984966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:38.985737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:38.986176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:39.316304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:39.317670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:39.324227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:39.324795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:39.325788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:39.326066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:39.326916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:39.337537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-58834219-4792-48c8-a0b0-e7e5bd7fc4e3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.363603 May 23 15:48:39.346933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 283/543] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:39.685670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:39.695995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:39.698057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:39.699557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77a79c34-8604-4808-ad95-0d1c86e9c0bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.387612 May 23 15:48:39.700228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 261/544] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:40.356731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:40.358321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:40.361870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=14 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:40.366486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:40.366933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:40.367767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:40.368698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:40.717860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:40.718632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:40.722858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:40.731037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:40.732118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:40.732860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b022b5d7-ea0c-422e-942e-aec767f8e7a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1273 microversion: 2.1 time: 0.380924 May 23 15:48:40.733477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 284/545] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1273 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:40.741146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-b1465d87-1381-4d1b-8155-bc6b52ebc7bc req-067169bd-c605-4aea-8dca-ab8244515373 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7915105-453d-453a-ad3c-656139c27d5a", "tag": "5f5a9b4b-5b24-4108-88fc-c41edee8bf44"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:40.744239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:40.745115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:40.750504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-b1465d87-1381-4d1b-8155-bc6b52ebc7bc req-067169bd-c605-4aea-8dca-ab8244515373 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:40.750954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-b1465d87-1381-4d1b-8155-bc6b52ebc7bc req-067169bd-c605-4aea-8dca-ab8244515373 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:40.762833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-b1465d87-1381-4d1b-8155-bc6b52ebc7bc req-067169bd-c605-4aea-8dca-ab8244515373 service nova] Creating event network-changed:5f5a9b4b-5b24-4108-88fc-c41edee8bf44 for instance c7915105-453d-453a-ad3c-656139c27d5a on hv-cinder-83588 May 23 15:48:40.768216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-b1465d87-1381-4d1b-8155-bc6b52ebc7bc req-067169bd-c605-4aea-8dca-ab8244515373 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033020 May 23 15:48:40.769308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 285/546] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:48:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:40.979684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:40.989309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:40.991039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:40.992357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7985f9de-af3d-4b7f-9a4c-4491a07f17d4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.278312 May 23 15:48:40.993168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 262/547] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:41.282137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-d75edd46-4704-41d1-ab3e-8635828eb17c req-92bc9bcf-f734-4244-926c-55f8a7fc72aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "name": "network-vif-plugged", "status": "completed", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:48:41.291923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-d75edd46-4704-41d1-ab3e-8635828eb17c req-92bc9bcf-f734-4244-926c-55f8a7fc72aa service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:41.292365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-d75edd46-4704-41d1-ab3e-8635828eb17c req-92bc9bcf-f734-4244-926c-55f8a7fc72aa service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:41.296308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:41.305244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-d75edd46-4704-41d1-ab3e-8635828eb17c req-92bc9bcf-f734-4244-926c-55f8a7fc72aa service nova] Creating event network-vif-plugged:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:48:41.310537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-d75edd46-4704-41d1-ab3e-8635828eb17c req-92bc9bcf-f734-4244-926c-55f8a7fc72aa service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032736 May 23 15:48:41.311141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 286/548] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:48:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:48:41.751602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:41.752322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:41.760798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:41.761564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:41.762438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:41.762774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:41.897834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:41.905736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-337e2505-9af0-4b7b-bc01-dc399858317b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.158406 May 23 15:48:41.906293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 263/549] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:42.013385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:42.014080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:42.021397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:42.021844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:42.022595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:42.023028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:42.158223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:42.166980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:42.168514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:42.169719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0237ae7-9fc3-4068-99eb-19b561aa55e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.160984 May 23 15:48:42.170271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 287/550] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:42.921736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:42.922347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:42.928815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:42.929259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:42.930010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:42.930441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:43.059466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:43.066875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-886f62a9-63b2-4d48-9c2a-36aa8affbe50 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.148933 May 23 15:48:43.067472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 264/551] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:43.188307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:43.188915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:43.196119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:43.196460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:43.197205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:43.197638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:43.330188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:43.338937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:43.340486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:43.341636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aa048dbb-2d0b-4e6f-84bd-2c9376d2deb8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.157266 May 23 15:48:43.342184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 288/552] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:44.083531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:44.084151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:44.090570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:44.091025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:44.091815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:44.092259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:44.227435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:44.234926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5c532af7-34de-4f8c-9130-cf6794609c40 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.155184 May 23 15:48:44.235748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 265/553] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:44.360210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:44.360772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:44.367205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:44.367655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:44.368414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:44.368845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:44.501238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:44.509333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:44.510840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:44.512030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee19385d-7df6-49a4-ba5a-787119c58c75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.155827 May 23 15:48:44.512585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 289/554] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:45.252674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:45.253290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:45.260547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:45.261002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:45.261766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:45.262208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:45.394120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:45.402584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3bb5d5e9-152d-4afd-9219-b02fec68b24c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.153715 May 23 15:48:45.403182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 266/555] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:45.531544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:45.532145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:45.539068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:45.539545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:45.540296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:45.540725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:45.669964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:45.678985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:45.680662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:45.681849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-16b122f1-0211-4207-9d43-b81a8e801e59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.154175 May 23 15:48:45.682392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 290/556] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:46.419560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:46.420117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:46.422375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:46.427765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:46.428211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:46.428973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:46.429405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:46.563821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:46.571326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4d2f2b8e-f0f8-422b-adcc-215c2aa2b02b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.155631 May 23 15:48:46.571892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 267/557] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:46.700365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:46.700911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:46.707936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:46.708382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:46.709141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:46.709574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:46.840414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:46.848353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:46.849860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:46.851015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-643ff03e-55b0-4d46-9bba-cebb912a96e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.154623 May 23 15:48:46.851596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 291/558] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:47.588275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:47.588889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:47.595437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:47.595893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:47.596681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:47.597121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:47.732861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:47.740747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b51c0d76-58ee-4595-86cc-2e23ad39b84d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.156264 May 23 15:48:47.741310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 268/559] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:47.869011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:47.869656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:47.876460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:47.876942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:47.877695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:47.878128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:48.009639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:48.017730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:48.019294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:48.020551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-215a571d-94b2-44d9-8a20-9ee895724824 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.155395 May 23 15:48:48.021627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 292/560] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:48.757608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:48.758220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:48.764850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:48.771857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:48.771857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:48.771857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:48.897312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:48.904782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a4bdc3b-5837-4fe3-9897-9cf255565cf7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.151008 May 23 15:48:48.909208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 269/561] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:49.039265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:49.039873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:49.046639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:49.047078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:49.047858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:49.048287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:49.177193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:49.185085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:49.186580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:49.187765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9380c22-73b8-442c-8b9d-ad7a3f414e2e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.152446 May 23 15:48:49.188311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 293/562] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:49.922538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:49.923175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:49.929727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:49.930179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:49.930945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:49.931409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:50.069448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:50.078381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f7efb0c9-9785-47c8-838d-4fced45bd13f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.159320 May 23 15:48:50.078739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 270/563] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:50.205886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:50.206487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:50.213682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:50.214117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:50.214867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:50.215327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:50.350067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:50.357865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:50.359402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:50.360562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03efafc2-8a2a-474c-bf34-0e6976a18105 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.158622 May 23 15:48:50.361116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 294/564] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:51.095779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:51.096319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:51.103546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:51.104206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:51.104965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:51.105396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:51.239367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:51.246999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4618c9a5-5c85-4f6e-97bb-04f6754ed824 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.155224 May 23 15:48:51.249118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 271/565] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:51.380556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:51.381143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:51.387598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:51.388047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:51.388807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:51.389245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:51.574046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:51.582903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:51.584387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:51.585677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-07ba3646-c7af-41d1-9411-b5a194af8c89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.209846 May 23 15:48:51.586218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 295/566] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:52.264233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:52.264893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:52.272084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:52.272528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:52.273282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:52.273713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:52.404691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:52.412300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3fce0a5d-a3bb-41a3-b6be-7c3ddcc959ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.151863 May 23 15:48:52.412864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 272/567] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:52.603945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:52.604704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:52.610967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:52.611438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:52.612191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:52.612619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:52.755027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:52.762868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:52.764388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:52.765533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2c1479e-82fc-4ae8-8623-5d0cdbc3e935 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.165566 May 23 15:48:52.766087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 296/568] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:53.429271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:53.429889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:53.436471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:53.436885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:53.437692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:53.438126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:53.442024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:53.570275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:53.578002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10cb3354-bbb8-4d74-8e49-0a8f9febbc41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.152523 May 23 15:48:53.578580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 273/569] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:53.784046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:53.784581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:53.790997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:53.791479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:53.792268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:53.792701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:53.929351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:53.929351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:53.929635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:53.929935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb370403-0ba4-48e4-917a-fc1699ae8816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.150064 May 23 15:48:53.930492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 297/570] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:54.595788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:54.596395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:54.603472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:54.603924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:54.604681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:54.605116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:54.742037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:54.749732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b953ef84-80ae-4cb4-a0b4-97f62955e79b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.157841 May 23 15:48:54.750316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 274/571] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:54.948694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:54.949228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:54.955627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:54.956354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:54.956818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:54.957254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:55.088057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:55.096129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:55.099360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:55.099360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b66fb4b9-5424-46c0-96ea-662f52999196 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.154074 May 23 15:48:55.099597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 298/572] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:55.765836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:55.766450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:55.773497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:55.773942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:55.774697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:55.775208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:55.907390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:55.915474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-210f41b5-9af0-49e2-8f11-4fc9cabb168b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.153472 May 23 15:48:55.916034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 275/573] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:56.117661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:56.118272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:56.125298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:56.125751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:56.126514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:56.126975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:56.257844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:56.265789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:56.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:56.268471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e64d1e4f-66ed-4f9d-ab96-b383a1abf829 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.155316 May 23 15:48:56.269032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 299/574] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:56.931475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:56.932090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:56.938802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:56.939292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:56.940055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:56.940490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:57.072028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:57.079460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f5478601-d887-4d62-a4df-679b8944e95f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.151945 May 23 15:48:57.080032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 276/575] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:57.286782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:57.287437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:57.293977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:57.294523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:57.295315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:57.295755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:57.425756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:57.434243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:57.435800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:57.436964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6e8f181-a9e1-4c35-b401-c1ff64ad1721 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.154257 May 23 15:48:57.437530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 300/576] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:58.096343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:58.097056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:58.103504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:58.103954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:58.104804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:58.105240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:58.236867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:58.245204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8f40f11e-027d-4789-9d68-912fe3b3c674 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.152576 May 23 15:48:58.245773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 277/577] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:58.455522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:58.456158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:58.462557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:58.463008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:58.463799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:58.464237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:58.595376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:58.604006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:58.605523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:58.606711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b031517b-c868-4f86-9099-c754b02bd972 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.155233 May 23 15:48:58.607285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 301/578] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:48:59.262180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:59.262825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:59.268796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:59.269236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:59.269988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:59.270418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:59.400050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:59.407575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2433a410-7b8b-4a33-99cb-82c2a118ab16 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.149807 May 23 15:48:59.408166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 278/579] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:48:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:48:59.625490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:48:59.626395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:48:59.632790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:59.633239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:59.641585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:48:59.641585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:48:59.765626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:59.774274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:59.775825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:48:59.776979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04595081-5e93-45d1-9209-171d4ea2f037 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.155692 May 23 15:48:59.777680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 302/580] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:48:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:00.425051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:00.425683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:00.432729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:00.433174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:00.433932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:00.434362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:00.563115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:00.570664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e527eea2-4641-46c5-9702-4b591acdac2d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.149511 May 23 15:49:00.571256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 279/581] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:00.795048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:00.795626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:00.802744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:00.803230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:00.804013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:00.804449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:00.822190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:00.935915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:00.944398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:00.948538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:00.948538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c33db8ec-d059-47cd-aa3c-552bc30a5b6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.156060 May 23 15:49:00.948538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 303/582] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:01.587611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:01.588213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:01.594807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:01.595300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:01.596071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:01.596508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:01.732706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:01.740778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0591d7b8-a37f-4560-b48b-195dcbf3bc11 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.156998 May 23 15:49:01.741348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 280/583] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:01.974599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:01.975238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:01.982020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:01.982500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:01.983291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:01.983751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:02.124507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:02.132749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:02.134367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:02.135542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e300c331-f8e3-478c-a035-84eb65e06aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.165423 May 23 15:49:02.136102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 304/584] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:02.759690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:02.760314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:02.770907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:02.771707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:02.772977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:02.773698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:02.911546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:02.918886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e2f8d8e9-7229-4d99-b6c9-3a37dd7fdfc5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.164753 May 23 15:49:02.919497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 281/585] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:03.153948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:03.154563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:03.161130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:03.161574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:03.162333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:03.162766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:03.292778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:03.300614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:03.302637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:03.303825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cbe59eed-c9ce-431a-a140-4eab0ee19784 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.153859 May 23 15:49:03.304500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 305/586] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:03.935823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:03.936441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:03.943633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:03.944087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:03.944851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:03.945290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:04.072104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:04.079469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b662e38-514d-43a8-8b82-28a5fb6bda58 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.147474 May 23 15:49:04.080032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 282/587] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:04.322353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:04.322885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:04.329418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:04.329865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:04.330618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:04.331061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:04.460208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:04.469267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:04.471088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:04.472373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-12fece1e-d9a0-46b4-a74f-730be6f24d64 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.153941 May 23 15:49:04.473491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 306/588] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:05.096256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:05.096774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:05.103807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:05.104328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:05.105009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:05.105441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:05.235816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:05.244039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c6df753b-e015-44b7-9a81-4dd4b54e30a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.151659 May 23 15:49:05.244608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 283/589] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:05.492250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:05.492864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:05.499687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:05.500138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:05.500904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:05.501339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:05.633310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:05.641140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:05.642744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:05.644032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb6f9af7-aa38-4dcd-b626-323a0dbc9fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.156778 May 23 15:49:05.644602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 307/590] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:06.261422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:06.262033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:06.268601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:06.269052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:06.269812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:06.270296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:06.399991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:06.407497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a7a31a5e-f12a-4507-9c7a-c1de21d72b8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.149841 May 23 15:49:06.408255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 284/591] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:06.662012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:06.662619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:06.664665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=58 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:06.669279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:06.669722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:06.670474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:06.670911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:06.800565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:06.813806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:06.815031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:06.816120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-729c655c-3bad-4d46-9ddc-f939629bd5af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1742 microversion: 2.20 time: 0.158188 May 23 15:49:06.817754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 308/592] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:06 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1742 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:07.424960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:07.425575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:07.433719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:07.434171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:07.434944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:07.435413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:07.581290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:07.591455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-afbff42f-d87c-4de4-9f9a-171864293b73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.169641 May 23 15:49:07.597641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 285/593] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:07.836729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:07.837488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:07.857453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:07.864537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:07.864537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:07.864537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:08.076237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:08.084399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:08.086005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:08.087424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2ad259f-54be-436f-a5e4-91c966c0a893 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.255015 May 23 15:49:08.088129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 309/594] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:08.608059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:08.609394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:08.618063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:08.618765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:08.620960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:08.621700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:08.829147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:08.836408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07e6af81-9cb1-486c-a8b9-9288c7b5b417 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.232070 May 23 15:49:08.837501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 286/595] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:09.107640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:09.109022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:09.116449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:09.117268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:09.118288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:09.119047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:09.278185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:09.293318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:09.295490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:09.296977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d96c404a-a007-46a0-92a0-eef46aa99d46 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.194557 May 23 15:49:09.307812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 310/596] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:09.854460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:09.855169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:09.861872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:09.862718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:09.863828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:09.864516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:09.999478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:10.007782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a990814-4990-4fd6-a92d-8cb05570b078 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.157201 May 23 15:49:10.008354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 287/597] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:10.317601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:10.318403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:10.326865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:10.327340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:10.328300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:10.329126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:10.484363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:10.493778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:10.495339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:10.496551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0de1591c-4a87-4a33-8d83-412f32a1f598 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.182929 May 23 15:49:10.497103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 311/598] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:11.023932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:11.024550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:11.031346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:11.032171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:11.032957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:11.033620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:11.197778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:11.206572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f414ec89-b176-4e3f-b2b4-bd377a2249c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.186431 May 23 15:49:11.207183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 288/599] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:11.258190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-706d0450-dd47-470c-b728-5dbbe43cce9b req-111ac314-de4e-4d11-aee0-dd0f00254ec0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "name": "network-vif-plugged", "status": "completed", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:11.277559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-706d0450-dd47-470c-b728-5dbbe43cce9b req-111ac314-de4e-4d11-aee0-dd0f00254ec0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:11.278277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-706d0450-dd47-470c-b728-5dbbe43cce9b req-111ac314-de4e-4d11-aee0-dd0f00254ec0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:11.303516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-706d0450-dd47-470c-b728-5dbbe43cce9b req-111ac314-de4e-4d11-aee0-dd0f00254ec0 service nova] Creating event network-vif-plugged:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 15:49:11.306018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-706d0450-dd47-470c-b728-5dbbe43cce9b req-111ac314-de4e-4d11-aee0-dd0f00254ec0 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054267 May 23 15:49:11.306938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 312/600] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:49:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:49:11.516873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:11.517624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:11.525754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:11.526201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:11.527023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:11.527487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:11.697627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:11.701382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:11.706470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:11.708070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:11.709248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdc27967-5be9-4e4e-ac86-023d92d0074a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.197791 May 23 15:49:11.710056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 289/601] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:12.224869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:12.225577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:12.237849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.238302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.239809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.240410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.398385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:12.406855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9bd46e42-ae26-4f48-98ff-178c2d5553a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.186947 May 23 15:49:12.407865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 313/602] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:12.515344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:12.515905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:12.522741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.523216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.523982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.524446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.656959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 45589704-9ca6-46fb-b1cd-8ba795032e49 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:49:12.729245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:12.729859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:12.735968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.736632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.737555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:12.738142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:12.887069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:12.896515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:12.898471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:12.900105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7814ae8c-b1e4-4947-a82f-2f3dcf58a5c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.174701 May 23 15:49:12.900973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 314/603] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:12.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d82ceff7-bc6c-4103-9ab6-c3e3ab4d94be tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49/os-volume_attachments/784d6bce-60ef-48f4-a3bd-817c1680b9ac" status: 202 len: 0 microversion: 2.1 time: 0.403438 May 23 15:49:12.915949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 290/604] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:49:12 2022] DELETE /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49/os-volume_attachments/784d6bce-60ef-48f4-a3bd-817c1680b9ac => generated 0 bytes in 406 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:49:13.424574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:13.425453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:13.433143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:13.433590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:13.434554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:13.435169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:13.585289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:13.593084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4fa7dd1a-0da6-42b4-8c85-673276291621 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.172479 May 23 15:49:13.593651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 315/605] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:13.920492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:13.921900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:13.928698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:13.929282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:13.930155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:13.930809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:14.069548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:14.078211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:14.079738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:14.080923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ccbb6b54-ac90-412e-9ad4-db0da2a15a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.165185 May 23 15:49:14.081470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 291/606] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:14.609764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:14.612526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:14.617953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:14.618448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:14.620683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:14.621329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:14.770599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:14.778322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-51858a1b-b12d-4423-9571-31ad23dac9f5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.172369 May 23 15:49:14.788115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 316/607] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:15.100308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:15.101731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:15.108902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:15.109576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:15.110917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:15.111872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:15.281880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:15.290831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:15.293878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:15.295414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e53f486-c971-4632-9c2c-76fcbeefc625 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.199345 May 23 15:49:15.307038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 292/608] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:15.795846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:15.796524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:15.803493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:15.804049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:15.804698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:15.805177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:15.940933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:15.948648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb21869b-23e8-4704-8216-8ac85509c32e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.156697 May 23 15:49:15.949224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 317/609] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:16.316100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:16.318182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:16.325245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:16.326580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:16.328252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:16.328811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:16.532827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:16.548070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:16.552890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:16.554626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cda64a71-d3b0-434d-8ee1-654b931302a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.243781 May 23 15:49:16.555506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 293/610] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:16.965873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:16.967192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:16.974465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:16.975202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:16.975953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:16.976577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:17.118605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:17.128785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-254086f4-4c63-4d8a-baad-c18c8cee5b86 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.166861 May 23 15:49:17.129632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 318/611] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:17.573169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:17.573893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:17.576479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=54 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:17.581328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:17.581769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:17.583061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:17.583519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:17.731993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:17.740573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:17.742088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:17.743290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6643c4b0-d565-42fd-9caa-91aa1c10a36e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.174056 May 23 15:49:17.743850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 294/612] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:17 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:18.157747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:18.158521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:18.165821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:18.166364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:18.167189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:18.167697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:18.368162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:18.376089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6aa9f929-8add-482f-91d0-086959abba71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.234208 May 23 15:49:18.377644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 319/613] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:18.762770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:18.763639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:18.771134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:18.771805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:18.772807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:18.773291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:18.960050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:18.968216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:18.969729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:18.971006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0051341-dd47-4aef-9c48-8c4639b1b710 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.212333 May 23 15:49:18.971826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 295/614] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:19.394158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:19.394738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:19.401886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:19.402342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:19.403159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:19.403797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:19.563923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:19.568014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:19.572753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-afe5707c-8bc8-4e3b-ade3-9f4eddb4e97d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.182695 May 23 15:49:19.573335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 320/615] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:19.993754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:19.994438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:20.001261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:20.001715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:20.002483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:20.002920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:20.133908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:20.141861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:20.143417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:20.144581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb4af937-57ee-4d25-96b9-746937b5f22e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.154821 May 23 15:49:20.145138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 296/616] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:20.589629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:20.590970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:20.601861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:20.603089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:20.603689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:20.604238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:20.764237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:20.773284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-93f78c92-83de-46c4-a390-ce0487aa1aa9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.187579 May 23 15:49:20.773859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 321/617] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:21.163774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:21.164385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:21.171127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:21.171573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:21.172331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:21.172762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:21.315301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:21.325282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:21.326841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:21.328039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8d45426e-a848-41bc-a1a0-0dcaf004d39e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.168143 May 23 15:49:21.331282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 297/618] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:21.789659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:21.790276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:21.796844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:21.797288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:21.798091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:21.798554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:21.938898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:21.946535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-38815dd9-aa6d-470f-8c42-18d304729433 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.161184 May 23 15:49:21.947136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 322/619] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:22.348018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:22.348790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:22.355623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:22.356072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:22.356867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:22.357305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:22.489324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:22.497610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:22.499691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:22.500973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4507a402-581b-4b56-aadb-edb4dd886161 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.156886 May 23 15:49:22.501670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 298/620] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:22.964638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:22.965189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:22.971894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:22.972338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:22.973109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:22.973707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:23.110564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:23.118090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0f36ef98-af8f-413c-bafb-94263c8561f9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.157418 May 23 15:49:23.118644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 323/621] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:23.520823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:23.521423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:23.528038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:23.528482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:23.529241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:23.529671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:23.664913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:23.673489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:23.675154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:23.676410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d58cc2a8-9515-47ab-ac45-22e4cea9b551 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.160001 May 23 15:49:23.677912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 299/622] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:24.136363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:24.137005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:24.143594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:24.144035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:24.144827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:24.145258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:24.283375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:24.291068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5509427-d5a2-45e3-b071-598fda272d63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.158630 May 23 15:49:24.291662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 324/623] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:24.695472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:24.696095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:24.703577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:24.704061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:24.704843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:24.705285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:24.835661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:24.844019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:24.847390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:24.847390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-020be514-df59-4b37-84dc-446ce324ef21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.155676 May 23 15:49:24.847390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 300/624] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:25.310419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:25.310951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:25.318055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:25.318505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:25.319299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:25.319812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:25.449877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:25.458042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-920196cd-4133-4e2a-8c8c-8354b2e8c402 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.152712 May 23 15:49:25.458604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 325/625] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:25.865542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:25.866152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:25.872222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:25.872671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:25.873430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:25.874117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:26.001412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:26.010131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:26.011752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:26.012912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-971b1d7a-01a7-44c5-b71f-3e34915a863d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.151903 May 23 15:49:26.013467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 301/626] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:26.477606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:26.478161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:26.484767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:26.485210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:26.485975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:26.486407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:26.626406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:26.633945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a814b043-3a4c-4f66-8012-79c4aa55bd02 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.162596 May 23 15:49:26.634525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 326/627] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:27.031855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:27.032396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:27.038807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:27.039294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:27.040065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:27.040561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:27.399923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:27.407713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:27.409301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:27.410966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d4301fe9-765d-443f-b680-2805eb783323 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.383123 May 23 15:49:27.411559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 302/628] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:27.650125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:27.650749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:27.657429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:27.657883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:27.658898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:27.659645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:27.794331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:27.802771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2ecb40f-6fd2-49d1-86d2-69b583adafc9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1261 microversion: 2.1 time: 0.156605 May 23 15:49:27.803574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 327/629] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:28.429439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:28.430054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:28.437628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.438092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:28.438878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.439344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:28.572636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:28.580837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:28.582497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:28.583699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53bce124-056d-45af-85be-beea0d223f23 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.158334 May 23 15:49:28.584268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 303/630] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:28.820688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:28.821262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:28.827941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.828387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:28.829266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.829698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:28.963721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:28.971630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-95af8bae-7f4a-4f4f-98e4-bd340ae47cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1444 microversion: 2.1 time: 0.155515 May 23 15:49:28.972200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 328/631] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1444 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:28.990052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:28.990748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:28.997111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.997466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:28.998250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:28.998958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.093813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:29.094304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:29.101327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.101773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.102528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.103148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.140448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:49:29.164925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:29.173897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-411bc954-d5d1-42cb-bbe3-ba8775f418a1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1444 microversion: 2.1 time: 0.187573 May 23 15:49:29.175208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 304/632] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1444 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:29.448549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0ba31ba-5987-4a4e-97c6-f667d0b1e340 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 204 len: 0 microversion: 2.1 time: 0.358333 May 23 15:49:29.449145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 329/633] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:49:29 2022] DELETE /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 15:49:29.458058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:29.458671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:29.465517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.466262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.466862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.467332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.601742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:29.603516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:29.604503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:29.605272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:29.610239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a6c1eb1-ce29-4aa5-9a50-f15496d9a4af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.156748 May 23 15:49:29.610824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 305/634] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:29 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:29.620104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.620636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.621451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:29.621889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:29.763587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:29.772906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:29.774545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:29.775814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-87515597-28b5-4842-a5fb-f9626b17f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.177098 May 23 15:49:29.776373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 330/635] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:30.628085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:30.628785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:30.635670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:30.636111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:30.636893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:30.637323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:30.782049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:30.790486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-47192321-93e9-43e3-8c69-80816e37a335 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.166626 May 23 15:49:30.791551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 306/636] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:30 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:30.801919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:30.801919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:30.802648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:30.803125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:30.803896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:30.804331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:30.947300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:30.955457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:30.957686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:30.958917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-06adc864-f251-4268-be11-84932c684f75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1737 microversion: 2.20 time: 0.168968 May 23 15:49:30.959507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 331/637] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:31.670149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-6b1e65bb-e670-41c3-b4f9-239937a5d3da req-ee0c0760-90bf-46b8-b5ca-97bcf6e37ea0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:31.679155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-6b1e65bb-e670-41c3-b4f9-239937a5d3da req-ee0c0760-90bf-46b8-b5ca-97bcf6e37ea0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:31.679686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-6b1e65bb-e670-41c3-b4f9-239937a5d3da req-ee0c0760-90bf-46b8-b5ca-97bcf6e37ea0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:31.691736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-6b1e65bb-e670-41c3-b4f9-239937a5d3da req-ee0c0760-90bf-46b8-b5ca-97bcf6e37ea0 service nova] Creating event network-changed:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 15:49:31.697460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-6b1e65bb-e670-41c3-b4f9-239937a5d3da req-ee0c0760-90bf-46b8-b5ca-97bcf6e37ea0 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031261 May 23 15:49:31.698069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 307/638] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:49:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:49:31.809378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:31.810113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:31.817124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:31.817583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:31.818359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:31.818798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:31.954504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:31.962437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-040436d8-82e0-4968-82a1-187efba17ca1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.156801 May 23 15:49:31.963022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 332/639] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:31 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:31.986261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:31.986964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:31.994021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:31.994475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:31.995283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:31.995725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:32.147811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:32.155936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:32.157463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:32.158660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4b4d2ec5-3520-4442-a66e-efc094405309 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.176602 May 23 15:49:32.164426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 308/640] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:32.979023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:32.979671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:32.986424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:32.986871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:32.987676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:32.988128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:33.132277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-2c4e0324-b7e1-4ecb-9729-eb12431bbfea req-37650850-c556-4c64-a597-6c1f1d205d3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "name": "network-vif-unplugged", "status": "completed", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:33.141364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2c4e0324-b7e1-4ecb-9729-eb12431bbfea req-37650850-c556-4c64-a597-6c1f1d205d3b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:33.141816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2c4e0324-b7e1-4ecb-9729-eb12431bbfea req-37650850-c556-4c64-a597-6c1f1d205d3b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:33.153462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-2c4e0324-b7e1-4ecb-9729-eb12431bbfea req-37650850-c556-4c64-a597-6c1f1d205d3b service nova] Creating event network-vif-unplugged:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:49:33.154320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:33.158336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-2c4e0324-b7e1-4ecb-9729-eb12431bbfea req-37650850-c556-4c64-a597-6c1f1d205d3b service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031370 May 23 15:49:33.158928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 309/641] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:49:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:49:33.164184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-290be855-c1d4-4226-8fa4-9a55d6a13095 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.189069 May 23 15:49:33.169861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 333/642] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:32 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:33.181216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:33.182036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:33.188654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:33.189110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:33.189882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:33.190319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:33.332038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:33.340032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:33.341581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:33.342749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52f15cfe-3cd0-4c06-83be-b3fa0fd10c4a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.168409 May 23 15:49:33.343375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 310/643] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:34.187296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:34.187927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:34.195179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:34.195635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:34.196405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:34.196846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:34.324460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:34.332599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9fc7194c-cc24-4ce6-9393-502b8cd20902 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.149104 May 23 15:49:34.336322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 334/644] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:34 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:34.360926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:34.361532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:34.367993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:34.368857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:34.369185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:34.369624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:34.512289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:34.521495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:34.523014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:34.524224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-26f8cb6a-47ce-4af2-bb7c-fcd4f3483604 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.167203 May 23 15:49:34.524782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 311/645] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:35.350237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:35.350853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:35.357557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:35.358003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:35.358799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:35.359438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:35.507416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:35.513519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:35.519831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-be115139-9972-4177-9a25-fe3f4ea5dd51 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.173315 May 23 15:49:35.520474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 335/646] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:35 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:35.542181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:35.542800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:35.549778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:35.550222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:35.551033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:35.551495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:35.682966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:35.690813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:35.692871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:35.694028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc905697-2023-45f7-9943-c42865cec2dd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.156268 May 23 15:49:35.694621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 312/647] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:36.539025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:36.539701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:36.548270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:36.548898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:36.549835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:36.550276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:36.712318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:36.712964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:36.714925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:36.719489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:36.719944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:36.720714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:36.721199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:36.722477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c871d5d-2a22-4a54-824b-16e71c29caa8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1514 microversion: 2.1 time: 0.188523 May 23 15:49:36.723568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 336/648] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:36 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1514 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:36.926145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:36.934750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:36.936560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:36.937748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-424f3831-6cde-43ef-8e30-b6021b467475 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.229465 May 23 15:49:36.938283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 313/649] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:37.197156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-99c2ab3e-accc-4726-9c13-02beb300fa9f req-bacdd45c-f620-4a6a-bc48-cc81d0942bdb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "45589704-9ca6-46fb-b1cd-8ba795032e49", "name": "network-vif-unplugged", "status": "completed", "tag": "9063491e-7ccf-41db-93ec-8b36f9d72163"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:37.206344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:37.209810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-99c2ab3e-accc-4726-9c13-02beb300fa9f req-bacdd45c-f620-4a6a-bc48-cc81d0942bdb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.210497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-99c2ab3e-accc-4726-9c13-02beb300fa9f req-bacdd45c-f620-4a6a-bc48-cc81d0942bdb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:37.224989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-99c2ab3e-accc-4726-9c13-02beb300fa9f req-bacdd45c-f620-4a6a-bc48-cc81d0942bdb service nova] Creating event network-vif-unplugged:9063491e-7ccf-41db-93ec-8b36f9d72163 for instance 45589704-9ca6-46fb-b1cd-8ba795032e49 on hv-cinder-83588 May 23 15:49:37.232295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-99c2ab3e-accc-4726-9c13-02beb300fa9f req-bacdd45c-f620-4a6a-bc48-cc81d0942bdb service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045167 May 23 15:49:37.234016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 337/650] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:49:37 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:49:37.739945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:37.740655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:37.747210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.747749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:37.748744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.749519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:37.821414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:37.825951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-81bb2eb8-f1a9-48e4-8bb7-d5dc51812fd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1472 microversion: 2.1 time: 0.090510 May 23 15:49:37.826780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 314/651] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:37 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1472 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:37.868256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-967f70d4-eb73-4203-9712-b3213aa13f1f req-77727e15-96cd-4706-9f5e-14b631e3f394 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "45589704-9ca6-46fb-b1cd-8ba795032e49", "name": "network-vif-deleted", "tag": "9063491e-7ccf-41db-93ec-8b36f9d72163"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:37.877036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-967f70d4-eb73-4203-9712-b3213aa13f1f req-77727e15-96cd-4706-9f5e-14b631e3f394 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.877481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-967f70d4-eb73-4203-9712-b3213aa13f1f req-77727e15-96cd-4706-9f5e-14b631e3f394 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:37.891660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-967f70d4-eb73-4203-9712-b3213aa13f1f req-77727e15-96cd-4706-9f5e-14b631e3f394 service nova] Creating event network-vif-deleted:9063491e-7ccf-41db-93ec-8b36f9d72163 for instance 45589704-9ca6-46fb-b1cd-8ba795032e49 on hv-cinder-83588 May 23 15:49:37.897534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-967f70d4-eb73-4203-9712-b3213aa13f1f req-77727e15-96cd-4706-9f5e-14b631e3f394 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034242 May 23 15:49:37.898144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 338/652] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:49:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:49:37.957752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:37.958204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:37.964535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.964980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:37.965750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:37.966187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:38.106441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:38.114372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:38.115925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:38.117087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2cce4c60-2115-4c8c-8fcc-47784d9372cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.164358 May 23 15:49:38.117647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 315/653] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:38.845290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:38.846279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:38.853333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:38.853847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:38.854878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:38.855736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:38.960698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:38.968389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-88394eb8-0fc8-4980-bcc8-3d438fd7e7fb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1472 microversion: 2.1 time: 0.128217 May 23 15:49:38.969365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 339/654] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:38 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1472 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:39.136314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:39.137012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:39.144000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:39.144726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:39.145550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:39.146033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:39.285465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:39.297583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:39.299408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:39.301094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1032d7c7-763b-4b15-937a-dad5d96096ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1719 microversion: 2.20 time: 0.168337 May 23 15:49:39.301658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 316/655] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1719 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:39.986483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:39.987665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:39.994633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:39.995245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:39.996119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:39.996664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:40.036953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:40.037716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:40.038396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'fault' on Instance uuid 45589704-9ca6-46fb-b1cd-8ba795032e49 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:49:40.087146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:40.094783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-11aeeaa0-7316-474f-b0a5-e1ea6a868a83 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1309 microversion: 2.1 time: 0.112290 May 23 15:49:40.096479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 340/656] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:39 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1309 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:49:40.320040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:40.320694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:40.328062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:40.328516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:40.336678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:40.337266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:40.559193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:40.567721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:40.569408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:40.570689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07309b0e-94ca-4273-bc6c-f1bf7c743638 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1722 microversion: 2.20 time: 0.254622 May 23 15:49:40.580655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 317/657] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1722 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:40.589121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "356edfc4-f82d-4a59-bd83-146727fd42c3"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:40.590279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:40.598532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:40.599072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:40.599942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:40.600664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:41.113986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:41.114881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:41.121390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:41.122096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:41.122915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:41.123385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:41.166382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:41.166844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:41.167390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'fault' on Instance uuid 45589704-9ca6-46fb-b1cd-8ba795032e49 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:49:41.263263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:41.271229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa595b44-8992-45f8-bb97-997d62f38299 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 200 len: 1309 microversion: 2.1 time: 0.161734 May 23 15:49:41.271819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 318/658] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:41 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 1309 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:49:41.886757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83ae06b9-bd07-4e9f-a888-416622cef76b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.300798 May 23 15:49:41.891506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 341/659] 10.222.0.32 () {62 vars in 1363 bytes} [Mon May 23 15:49:40 2022] POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 188 bytes in 1304 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:49:42.067941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:42.070506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:42.075469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.076149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.076991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.077497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.292596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:42.292596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 45589704-9ca6-46fb-b1cd-8ba795032e49] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:42.295718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.296375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.297341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.297932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.313123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 HTTP/1.1 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:49:42.315027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. May 23 15:49:42.317193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '45589704-9ca6-46fb-b1cd-8ba795032e49'} May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:49:42.317921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. May 23 15:49:42.319206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:49:42.321997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 404 to user: Instance 45589704-9ca6-46fb-b1cd-8ba795032e49 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:49:42.323089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fceb469d-7569-4789-a2df-73792016829a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49" status: 404 len: 111 microversion: 2.1 time: 0.039333 May 23 15:49:42.323881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 342/660] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:49:42 2022] GET /compute/v2.1/servers/45589704-9ca6-46fb-b1cd-8ba795032e49 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 15:49:42.408325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-efb20fc1-1425-4800-906d-9e79b2d03118 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action" status: 202 len: 0 microversion: 2.20 time: 0.345622 May 23 15:49:42.408941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 319/661] 10.222.0.32 () {62 vars in 1318 bytes} [Mon May 23 15:49:42 2022] POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action => generated 0 bytes in 347 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 15:49:42.421237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:42.421887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:42.441831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.441831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.442558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:42.443013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:42.742326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:42.755128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:42.756893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:42.763127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9c5aebe8-f359-4704-a20e-5527f77395ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1748 microversion: 2.20 time: 0.348790 May 23 15:49:42.768365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 343/662] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1748 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:43.785918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:43.786557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:43.793268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:43.793768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:43.794586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:43.795028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:43.997712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:44.008989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:44.010857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:44.012398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a3f602d-6886-4832-aeb0-7b007c91cdb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1748 microversion: 2.20 time: 0.230503 May 23 15:49:44.013307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 320/663] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1748 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:45.031710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:45.032256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:45.040128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:45.040777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:45.041605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:45.042228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:45.227363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:45.246678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:45.248230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:45.250780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3e348631-7164-49ec-916b-0e0520d77860 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1748 microversion: 2.20 time: 0.222514 May 23 15:49:45.250780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 344/664] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1748 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:46.270638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:46.271480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:46.278380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:46.278831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:46.279672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:46.280105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:46.416790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:46.432398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:46.435063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:46.436457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-820dc7a8-ea54-40e1-9d46-4dab598d510e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1748 microversion: 2.20 time: 0.170599 May 23 15:49:46.437455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 321/665] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1748 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:47.455705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:47.456315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:47.462957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:47.470120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:47.470120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:47.470120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:47.767804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:47.777987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:47.779935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:47.782950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-25c26021-d98c-4ed6-b731-e95611791a19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1748 microversion: 2.20 time: 0.330847 May 23 15:49:47.784565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 345/666] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1748 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:48.807231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:48.807231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:48.811945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=60 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:48.814276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:48.814981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:48.816879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:48.817582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:48.862652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:49.022637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:49.031750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:49.033754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:49.034963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ac55506a-79ea-4a6e-bf5b-e1930ca57193 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1750 microversion: 2.20 time: 0.237028 May 23 15:49:49.040381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 322/667] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1750 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:50.059134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:50.060350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:50.067478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:50.068247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:50.069187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:50.069787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:50.322274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:50.333294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:50.335325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:50.336617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-35fc025d-9b7f-4bb2-986d-9057d9367dbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.281471 May 23 15:49:50.338204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 346/668] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:51.358814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:51.360187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:51.374326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:51.375045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:51.375945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:51.376740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:51.570440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:51.580430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:51.584008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:51.585601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-863668ab-da97-4fa7-aa51-7e3c0d6cd68d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.232887 May 23 15:49:51.586423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 323/669] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:52.605004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:52.606394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:52.614014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:52.614754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:52.623433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:52.624387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:52.800240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:52.808570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:52.810139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:52.813323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-232f1d58-7128-4e6d-b3ee-83e7296cb0ca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.212353 May 23 15:49:52.814442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 347/670] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:53.553960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:53.554504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:53.576297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:53.576767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:53.577619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:53.583400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:53.662791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae162603-6dca-44c5-a057-d50d35c3aed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/ips" status: 200 len: 147 microversion: 2.1 time: 0.114510 May 23 15:49:53.663959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 324/671] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:49:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/ips => generated 147 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:49:53.839713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:53.840339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:53.864025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:53.864486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:53.865275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:53.865960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:54.322691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:54.322691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:54.325327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:54.325327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5b2d907-7303-4183-a088-24cccbec1685 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.442181 May 23 15:49:54.325327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 348/672] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:55.342372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:55.350658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:55.357976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:55.358433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:55.359250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:55.359693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:55.575784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-342115805", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "networks": [{"uuid": "fde7f6c3-1fe4-4f6b-9aa5-195b397c0f47"}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:49:55.733795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:55.746218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:55.748337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:55.749977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0de32667-246e-49cf-bef2-ef032cae13ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.415192 May 23 15:49:55.751029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 325/673] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:49:56.773782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:56.775477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:56.782673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:56.783240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:56.784278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:56.784904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:56.979623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:56.990189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:56.991728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:56.993313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa3c9f90-133b-4b43-a190-6d6075d88e70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.227556 May 23 15:49:56.994333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 326/674] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:58.016955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:58.017679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:58.025337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:58.025894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:58.027522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:58.028556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:58.179260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:58.187867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:58.189378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:58.190543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-47f9e070-7f00-43c8-8960-e99dfb1d6542 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.177871 May 23 15:49:58.191855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 327/675] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:49:59.210982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:49:59.212584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:49:59.219534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:59.220214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:59.221583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:49:59.222490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:49:59.467975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:59.467975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:59.468568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:49:59.468568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b514013-ed64-4d7c-87fb-76ae310b360e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.187680 May 23 15:49:59.468568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 328/676] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:49:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:00.414446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:00.416527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:00.432046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:00.432888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:00.433933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:00.434594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:00.588976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:00.601951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:00.603647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:00.604960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15de52d1-a18a-41dc-b97d-17b5bfc6a594 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.195436 May 23 15:50:00.616583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 329/677] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:00.622045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f022774e-0ec3-444b-af35-2dc0e23213e3"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:00.625568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:00.632401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:00.632966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:00.633833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:00.634327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:02.962874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] validate_networks() for [('fde7f6c3-1fe4-4f6b-9aa5-195b397c0f47', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:50:03.317514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:50:03.317962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:50:03.318586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:50:03.319088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:50:03.319527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:50:03.329915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:03.330371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:03.331210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:03.331647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:03.348769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Getting quotas for project 7e22027c2e5948a6ba154c837a7673a9. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:50:03.352657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Getting quotas for user cc4c3405e93141fc851f22e49eee09bd and project 7e22027c2e5948a6ba154c837a7673a9. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:50:03.363953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:50:03.366231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:50:03.531853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56cdf738-e8e9-49f3-9d04-9d58cd705ada tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 7.960084 May 23 15:50:03.533392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 349/678] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:49:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 7962 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:50:03.537769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:03.538430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:03.544848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:03.545307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:03.546100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:03.547236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:03.705959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:03.719054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:03.720876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:03.722099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36ca3633-87a8-4189-a1dd-8748c65be191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.188396 May 23 15:50:03.722693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 350/679] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:03.737258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-feb0d758-ed11-43cb-9edb-af899db15c9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'create', calling method: .version_select at 0x7f47f912c1f0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1208197562", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLzSEPX/+RhezYqey/PR9mkwCwACXat5YtZKTx1Vsz9s6bTFZxO2cJ02goJpj3mcOKrztasYYLbAoxA7KbxWGhkldrl6ELmDd4gHOXAC+ybaQBZeufN+Hktk16/greiYUQ=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:03.754680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-feb0d758-ed11-43cb-9edb-af899db15c9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Getting quotas for project 8ce8b36a28714d618becacd8e2925a75. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:50:03.759156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-feb0d758-ed11-43cb-9edb-af899db15c9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Getting quotas for user 16a2ff4a80da41bf84f19450000a9da1 and project 8ce8b36a28714d618becacd8e2925a75. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:50:03.897694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-feb0d758-ed11-43cb-9edb-af899db15c9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.170210 May 23 15:50:03.898319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 351/680] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 15:50:03 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:03.903139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9b576a8b-fdd1-4ecf-831a-0c3749a2acd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:03.903766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9b576a8b-fdd1-4ecf-831a-0c3749a2acd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:03.961017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9b576a8b-fdd1-4ecf-831a-0c3749a2acd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:03.968717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9b576a8b-fdd1-4ecf-831a-0c3749a2acd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1218 microversion: 2.1 time: 0.069200 May 23 15:50:03.969946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 352/681] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1218 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:03.974506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-985776537", "imageRef": "88354df0-750d-4f6e-96e5-6d6e93358b08", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1208197562", "networks": [{"uuid": "a1bc1dbc-ee6f-43d5-b155-cca78e3710dd"}]}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:04.022670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] validate_networks() for [('a1bc1dbc-ee6f-43d5-b155-cca78e3710dd', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 15:50:04.432817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 15:50:04.433242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 15:50:04.433719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 15:50:04.434127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 15:50:04.434538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 15:50:04.445111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:04.445566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:04.446494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:04.446942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:04.466627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Getting quotas for project 8ce8b36a28714d618becacd8e2925a75. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 15:50:04.470492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Getting quotas for user 16a2ff4a80da41bf84f19450000a9da1 and project 8ce8b36a28714d618becacd8e2925a75. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 15:50:04.487264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 15:50:04.490021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='88354df0-750d-4f6e-96e5-6d6e93358b08',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 15:50:04.573972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:50:04.574628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 15:50:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments (ip 10.222.0.32) !!! May 23 15:50:04.683513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee93fb14-5599-46b6-84a5-743e2d6f70e8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.712177 May 23 15:50:04.684122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 353/682] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 15:50:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 714 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 15:50:04.694679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-887695e7-706c-47cc-8d64-4cd99ce56ff6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:04.697713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-887695e7-706c-47cc-8d64-4cd99ce56ff6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:04.756083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-887695e7-706c-47cc-8d64-4cd99ce56ff6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:04.767915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-887695e7-706c-47cc-8d64-4cd99ce56ff6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1258 microversion: 2.1 time: 0.077785 May 23 15:50:04.777591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 354/683] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1258 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:04.917199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:04.917847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:04.924222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:04.924668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:04.926019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:04.926456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:05.061092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:05.064669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:05.069810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:05.071327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:05.072478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dc6e6d88-3ba4-433c-997a-b88d998af483 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.303030 May 23 15:50:05.073025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 355/684] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:05.077375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:05.077975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:05.084021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:05.084478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:05.085338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:05.085805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:05.153545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:05.161759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-240cd3fe-b95d-4e9c-9fc2-9e32ead077b9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.087758 May 23 15:50:05.162338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 356/685] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:05.785310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9939992f-1554-4c1d-96cc-7a8f439beb9d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:05.798814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9939992f-1554-4c1d-96cc-7a8f439beb9d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:05.845077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9939992f-1554-4c1d-96cc-7a8f439beb9d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:05.851596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9939992f-1554-4c1d-96cc-7a8f439beb9d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1258 microversion: 2.1 time: 0.070288 May 23 15:50:05.852149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 357/686] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1258 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:06.092177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:06.092810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:06.099026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.099508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.100271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.100713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.243564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:06.256516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:06.258061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:06.259333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2e55775-af34-4d18-b646-4c0415dc0910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.171785 May 23 15:50:06.262417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 358/687] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:06 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:06.266842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:06.267488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:06.273807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.274258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.275040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.275549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.350467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:06.356837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ccc2bbfa-573b-49a0-bc89-7dd653a4940f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.093398 May 23 15:50:06.357649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 359/688] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:06.869520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:06.870612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:06.880080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.880895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.882238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:06.882886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:06.952656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:06.960595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6bee28b-9a10-46e0-8013-7746747ecbfd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.095577 May 23 15:50:06.961177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 360/689] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:06 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:07.278238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:07.278918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:07.285623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.286070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:07.286825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.287286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:07.427872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:07.436258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:07.437782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:07.439000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dd3abc1f-7305-4d78-a673-9fcdd30dbaa4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.164896 May 23 15:50:07.439603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 361/690] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:07.445147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:07.446046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:07.451739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.452187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:07.452947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.453384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:07.520671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:07.522861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-13c65b1a-2f49-4962-9da2-8de251e8283c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.082001 May 23 15:50:07.523473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 362/691] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:07.978412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:07.978907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:07.981204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:07.986105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.986571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:07.987429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:07.988467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:08.049354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:08.057775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e5f1b18-b874-48db-9ed3-a351e6544b93 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.083283 May 23 15:50:08.058363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 363/692] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:08.463216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:08.463890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:08.469941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:08.470379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:08.471145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:08.471580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:08.607200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:08.615602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:08.617107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:08.618273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e5d0b49-faeb-4edc-bf0b-186a18f15799 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.159314 May 23 15:50:08.618821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 364/693] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:08.623334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:08.623939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:08.629984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:08.630434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:08.631223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:08.631671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:08.692352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:08.700127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2bd52581-b5e2-42d6-8299-29cfba472b64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.080208 May 23 15:50:08.700717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 365/694] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:09.074855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:09.075494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:09.082280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.082734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.083564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.084001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.152592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:09.166383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4881059-510e-4072-a7da-a9e187756413 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.095212 May 23 15:50:09.167210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 366/695] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:09.637147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:09.637881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:09.648138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.648593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.649875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.650337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.802968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:09.814361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:09.816614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:09.818316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7ba8239-cb23-4860-adcf-128b97213725 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.185041 May 23 15:50:09.819291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 367/696] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:09.829315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:09.830164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:09.837770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.838211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.838984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:09.839437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:09.905362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:09.912803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-43923c8f-b8f5-4539-ba13-1518617e527f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.091290 May 23 15:50:09.913362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 368/697] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:10.184044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:10.184577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:10.191025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:10.191504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:10.192253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:10.192684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:10.265372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:10.266384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d59f3af-795d-43c6-804d-24557a56d816 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.086227 May 23 15:50:10.266965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 369/698] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:10.632059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=17 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:10.717874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=48 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:10.838272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:10.838845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:10.845104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:10.845558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:10.846336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:10.846779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:10.997613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:11.009351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:11.018069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:11.018069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-320a0ad6-b865-4b3e-b48e-4869e52dd2c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.178446 May 23 15:50:11.018069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 370/699] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:11.021082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:11.021701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:11.027845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:11.028294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:11.029062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:11.029599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:11.090981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:11.098739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a04ce10-f0e7-42bf-9fee-e8798264bdbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.081874 May 23 15:50:11.099446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 371/700] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:11.283257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:11.283931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:11.290657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:11.291179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:11.291906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:11.292343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:11.354888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:11.361395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ac55abb8-73a0-42f5-beec-991eacae9d91 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.082022 May 23 15:50:11.361944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 372/701] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:12.032737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:12.033406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:12.039579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.040022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.040778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.041212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.060047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:12.178161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:12.186889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:12.188408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:12.189580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-088e6dbb-6135-46a9-b858-c7db3f2324a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.161006 May 23 15:50:12.201063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 373/702] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:12.201063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:12.201063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:12.204475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.204928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.205702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.206133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.270941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:12.278493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff8e66e4-6985-4a4f-94ed-79b059096a13 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.087367 May 23 15:50:12.279047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 374/703] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:12 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:12.380841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:12.381447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:12.387942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.388385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.389130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:12.389559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:12.450261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:12.457719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17f135a6-40f3-456d-aa4b-2c5ae5b994bd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.082149 May 23 15:50:12.458308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 375/704] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:13.208776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:13.209646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:13.218116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.218886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.220236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.220665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.356626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:13.366113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:13.367671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:13.368822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-249ce4b1-a183-459b-9979-3387c3f43073 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.164176 May 23 15:50:13.369403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 376/705] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:13.376887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:13.377491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:13.383339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.383783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.384535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.384964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.453131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:13.460523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9c2fc9b-687b-492f-b326-a735db8e3451 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.090182 May 23 15:50:13.461081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 377/706] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:13.474521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:13.475157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:13.481003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.482204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:13.482638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:13.546942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:13.558474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-648cbef5-b142-4d2f-b37e-2d1bf918805f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.084356 May 23 15:50:13.558474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 378/707] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:13 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:14.387701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:14.393389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:14.395908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.396365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.397130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.397606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.455842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:14.527692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:14.536796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:14.538325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:14.539817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d246ba90-bb76-4c12-9408-4f10262ef68e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.155656 May 23 15:50:14.541352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 379/708] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:14.546386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:14.547128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:14.553060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.553585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.554466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.554895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.619960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:14.627550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7f1260ec-1b47-4dda-acbe-0ece8b916075 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.085228 May 23 15:50:14.628114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 380/709] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:14.632662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:14.633261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:14.639739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.640278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.641036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:14.641470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:14.702405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:14.711946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24a8e9fa-9816-4ac1-be34-4b0f775b0ef9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.082711 May 23 15:50:14.712627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 381/710] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:14 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:15.559175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:15.559837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:15.566836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.567672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.568842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.569406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.710679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:15.719377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:15.721046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:15.722261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c94c83a-82c5-4dbd-b53d-10aecad75a75 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.166999 May 23 15:50:15.722862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 382/711] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:15.727428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:15.728050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:15.734131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.734592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.735573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.736018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.812567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:15.820584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-94189b10-2c84-4e1f-998e-bfaca0817ed4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.096747 May 23 15:50:15.821145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 383/712] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:15.826311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:15.826850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:15.833153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.833599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.834361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:15.834797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:15.849438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=22 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:15.902227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:15.909753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0392bb5c-e47c-4df8-ae02-6fe280259b1c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.087653 May 23 15:50:15.912795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 384/713] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:16.745078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:16.745741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:16.752494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:16.752936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:16.753696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:16.754738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:16.896644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:16.905352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:16.906844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:16.908051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-88e9b2da-27d4-4859-aa9f-e85b8a7445c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.166872 May 23 15:50:16.908611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 385/714] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:16.914041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:16.914558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:16.920715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:16.921167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:16.921923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:16.922351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:16.983243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:16.991267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-997b0e5c-39aa-4d39-9e43-f09ab28497f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.081666 May 23 15:50:16.991827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 386/715] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:16 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:16.996240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:16.996851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:17.002840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:17.003326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:17.004115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:17.004551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:17.065752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:17.073239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cbfa6eae-96f1-43c1-acf1-8c4b3f3ec317 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.080475 May 23 15:50:17.073794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 387/716] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:17.927039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:17.927728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:17.934136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:17.934579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:17.935360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:17.935795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:18.074805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:18.084211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:18.085708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:18.086868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2270b4a5-6bf2-41a8-b53c-58a726840e9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.163861 May 23 15:50:18.087460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 388/717] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:17 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:18.091775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:18.092383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:18.098172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:18.098626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:18.099433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:18.099877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:18.162965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:18.170280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-818d0e9a-f4f9-46e0-9ef9-0044bebf91a2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.081895 May 23 15:50:18.170837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 389/718] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:18.175487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:18.176584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:18.182456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:18.182842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:18.183637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:18.184072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:18.187942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:18.244083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:18.251728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e933673-1efd-4bdf-b94a-5a9429539188 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.079917 May 23 15:50:18.252286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 390/719] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:18 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:19.114913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:19.114913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:19.114913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.114913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.116047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.116047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.255197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:19.255991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:19.257490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:19.258648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4d0b97e4-62ca-48cd-b7b8-eee160521446 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.157512 May 23 15:50:19.259257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 391/720] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:19.263694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:19.264484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:19.270707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.271184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.271940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.272374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.334396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:19.341922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7090fcf-503a-495a-8910-ba6382932af6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1222 microversion: 2.1 time: 0.081698 May 23 15:50:19.342474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 392/721] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1222 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:19.348511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:19.348859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:19.355488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.356900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.357371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:19.357802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:19.420926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:19.429553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a0587f42-a23e-4e16-aa5a-a5d68c2eceec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1262 microversion: 2.1 time: 0.085557 May 23 15:50:19.429917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 393/722] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1262 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:20.042336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-09ee81ec-e947-4764-b817-77d93cdb947f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 19.424073 May 23 15:50:20.043052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 330/723] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:50:00 2022] POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 194 bytes in 19426 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:20.277132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:20.277747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:20.284258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.284711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.286129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.286561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.359198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:20.360158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:20.367935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.367935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.367935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.368308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.452565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:20.462033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-34bf5693-31fc-43e1-9a81-6a97e203ad74 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1214 microversion: 2.1 time: 0.106933 May 23 15:50:20.462620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 331/724] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:20 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1214 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:20.464610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:20.467893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:20.469185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:20.475868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.476314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.477091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:20.477530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:20.479403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:20.481025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:20.482370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08053260-c264-4d40-a4cf-16871e5ba504 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.209185 May 23 15:50:20.483082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 394/725] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:20.542273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:20.549707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-86916897-1037-41e9-bbf3-81328afbab73 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1254 microversion: 2.1 time: 0.086105 May 23 15:50:20.550288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 332/726] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:20 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1254 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:21.479355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:21.479979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:21.490162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.492109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.492109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.493130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.511780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:21.512541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:21.521665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.522418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.523498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.524123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.580844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:21.592522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-069eb2b7-5f2a-4c98-8823-41f5942f43eb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1270 microversion: 2.1 time: 0.117119 May 23 15:50:21.593149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 395/727] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1270 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:21.598329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:21.600064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:21.605592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.606047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.606831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:21.607344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:21.701572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:21.709920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9beafa4e-1286-4e7c-8c67-49c0ca05b8cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1318 microversion: 2.1 time: 0.114227 May 23 15:50:21.710802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 396/728] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:21 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1318 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:21.737049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:21.746028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:21.748797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:21.749065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-652a9cfa-50ed-482d-82e0-1c5bd51ea4b9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.246401 May 23 15:50:21.751718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 333/729] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:22.610515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:22.611195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:22.629240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.629949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:22.630926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.631542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:22.730435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:22.732173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:22.743637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.745185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:22.747004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.749850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:22.777356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:22.792490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a34f1a3d-cb65-4909-be56-d4c3cf3947c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1288 microversion: 2.1 time: 0.186414 May 23 15:50:22.813323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 397/730] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1288 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:22.818781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:22.824809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:22.850121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.850831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:22.851860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:22.852452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:23.116002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:23.132792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:23.134954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:23.144869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff49e336-1229-4145-a3ed-c27120ca5c7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.333100 May 23 15:50:23.147428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:23.150560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 398/731] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:23.161010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4948813a-8fbb-4ce1-98e5-5eaec8a2b76f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.434498 May 23 15:50:23.161811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 334/732] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:23.821881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:23.822485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:23.829079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:23.829529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:23.830293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:23.830723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:23.976060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:23.989071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ecdfc75-2952-406e-8a22-f0993dd15c8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.171082 May 23 15:50:23.989638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 399/733] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:24.169681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:24.170275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:24.177065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:24.177615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:24.179145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:24.179622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:24.182759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:24.183544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:24.189433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:24.189876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:24.190639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:24.191069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:24.387526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:24.390775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:24.404398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:24.405892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:24.408540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe03524d-99e0-4f39-83c3-bdec406713b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.229579 May 23 15:50:24.409209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 400/734] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:24.411196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10dd4182-7112-4d57-8059-6ef9b59e52bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.245229 May 23 15:50:24.412105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 335/735] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:25.011915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:25.012661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:25.019630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.020015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.020803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.022060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.164450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:25.172102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20cf4e1a-5a5c-423f-a6bd-aae2d458e955 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.164282 May 23 15:50:25.173009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 401/736] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:25 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:25.427772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:25.429120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:25.432961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:25.432961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:25.436692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.437240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.437556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.438107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.438240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.438691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.438883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:25.439352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:25.653057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:25.664950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:25.667277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ffe1b50-0fa1-4559-b569-0489a6eba6e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.244987 May 23 15:50:25.668282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 336/737] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:25.673852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:25.675398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:25.676640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d0030c7-4d19-4da6-8989-d8a5df8b314a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.250849 May 23 15:50:25.677193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 402/738] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:26.193054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:26.193666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:26.199948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.200402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.201176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.201612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.329673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:26.337844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-94caa9b1-4940-467a-bf98-1aa97fc9b5a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148589 May 23 15:50:26.338416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 337/739] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:26 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:26.695575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:26.696260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:26.699550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:26.700234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:26.704902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.705357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.706135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.706563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.706794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.707335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.708742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:26.709181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:26.903673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:26.911878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:26.912689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:26.914407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:26.915637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5607ea9a-f6bb-4c43-9685-e939114f0964 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.224829 May 23 15:50:26.916540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 338/740] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:26.930492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f07682fe-b715-46ae-9986-6e4b352279c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.244518 May 23 15:50:26.931435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 403/741] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:27.355023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:27.355668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:27.361804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.362255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:27.363023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.363992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:27.508363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:27.511671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-baee7239-1844-4fc0-8873-8fcdb5d8e6b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.160523 May 23 15:50:27.512246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 339/742] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:27.938721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:27.939373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:27.947601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.947816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:27.948264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:27.948347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:27.948811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.949251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:27.955540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.955996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:27.956777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:27.957755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:28.157817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:28.161720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:28.166767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:28.168309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:28.169313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b53c8d0e-45d4-488f-9aa5-9d0e58e50ee8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.225508 May 23 15:50:28.169475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f3ccf552-1ade-498f-a629-1ea454d7c923 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.236947 May 23 15:50:28.169890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 340/743] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:27 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:28.170048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 404/744] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:28.528975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:28.529585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:28.536489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:28.536939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:28.537709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:28.538142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:28.672195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:28.681033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a73330f-80a1-4b4f-aadb-faf5babace19 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.155791 May 23 15:50:28.681651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 341/745] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:29.186918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:29.187759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:29.189385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:29.190071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:29.196982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.197439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.198316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.198656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.202517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.203227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.203999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.204431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.646631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:29.651058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:29.656196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:29.658192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:29.659584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a380063d-6476-49ce-ade0-8f2ff94c654c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.476849 May 23 15:50:29.660183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f983baff-3590-4698-b42b-6bb5dc59b753 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.474461 May 23 15:50:29.660883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 342/746] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:29.661404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 405/747] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:29.700910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:29.701884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:29.709067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.709531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.710323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:29.710763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:29.871287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:29.879709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5516917b-86c2-418c-995c-1f3684887ffd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.184027 May 23 15:50:29.880462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 343/748] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:30.678259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:30.678874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:30.679642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:30.680257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:30.689922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.690377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.690496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.690812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.691157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.691609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.691729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.692046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.886404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:30.889911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:30.895230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f014adf5-9b2b-45f9-8bde-5459892909a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.220619 May 23 15:50:30.895983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 406/749] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:30.900430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:30.901247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:30.907331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:30.908191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.908759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.909629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:30.910180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:30.916506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:30.918017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:30.919309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-41aba321-cc57-4037-987b-631be2e2149f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.245571 May 23 15:50:30.919877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 344/750] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:31.049792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:31.057867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89015106-28df-43ce-890e-41f6125703ab tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.160938 May 23 15:50:31.058540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 407/751] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:30 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:31.914116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:31.914782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:31.922200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:31.922694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:31.923526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:31.923958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:31.940506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:31.940506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:31.947609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:31.948637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:31.949615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:31.950234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:32.156749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:32.161780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:32.165395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d623fa1f-c8cf-4696-9e58-2302bab89d2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.255500 May 23 15:50:32.171267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:32.172872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:32.174124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f38d93e1-ba05-4608-898e-dc08e20b8bb1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.240309 May 23 15:50:32.174758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 345/752] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:32.175005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 408/753] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:32.190121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:32.190729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:32.197558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:32.198008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:32.199066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:32.199210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:32.204492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:32.339969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:32.348531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da2b7fdf-e15f-4200-8ff7-f30063692e82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.161529 May 23 15:50:32.349103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 346/754] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:33.184263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:33.184882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:33.193662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:33.194422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:33.195737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.196498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.197439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.198094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.202091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.203261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.203407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.203804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.401823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:33.405033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:33.412318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5e40f73-6ec5-4d11-975c-94d1c0113177 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.233416 May 23 15:50:33.417277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 409/755] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:33.417421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:33.417421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:33.417619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:33.418196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:33.418481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9361ded-45f6-436a-8106-c508420ca21f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.229254 May 23 15:50:33.419201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 347/756] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:33.424493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.424946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.425732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:33.426166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:33.574887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:33.582499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c84bb3d4-3f11-407e-939e-24109192fca9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.168586 May 23 15:50:33.583706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 410/757] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:33 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:34.430318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:34.430932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:34.437095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:34.437577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.437687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:34.437922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.438693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.439338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.444340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.444831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.445588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.446022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.656175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:34.659231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:34.664554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ecf2601-c5ef-4ff5-bbde-48a7469eed8b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.238611 May 23 15:50:34.665230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 348/758] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:34 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:34.669019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:34.670216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:34.670518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:34.670958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:34.671719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3028e8c4-8ab9-44f4-ade1-2efcd859441e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.238480 May 23 15:50:34.672512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 411/759] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:34.678246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.678696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.679512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:34.679947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:34.817049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:34.825265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b4bdd8e6-0606-41a7-a95c-8ff59f4d71fc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.158976 May 23 15:50:34.825852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 349/760] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:35.682416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:35.683021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:35.689814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.690263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:35.691025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.691377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:35.691860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:35.691999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:35.698877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.699398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:35.700151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.700586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:35.895371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:35.898895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:35.903014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-92193dfb-bff1-41ae-893b-d32b28b643f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.224414 May 23 15:50:35.903614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 412/761] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:35.907469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:35.908012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:35.908615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:35.909059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:35.910301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f9b9b7e0-8b8b-4031-a369-5ffced97cea3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.223090 May 23 15:50:35.910480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:35.913180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 350/762] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:35.916673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.917206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:35.918041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:35.918714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:36.054013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:36.061452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-521a51ba-ee11-4cad-ae46-f77f59e0679f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.156844 May 23 15:50:36.061940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 413/763] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:35 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:36.925843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:36.926989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:36.928669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:36.929365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:36.934900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:36.935514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:36.936907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:36.937374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:36.937507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:36.938329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:36.938578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:36.939058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:37.174709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:37.178487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:37.183352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da595fcc-e53e-48a0-89da-8eaa0dcc7f68 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.263185 May 23 15:50:37.183929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 351/764] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:37.187000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:37.188598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:37.188747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:37.189239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:37.189764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a08e4ca-0902-463d-9c0a-721bff9c2f36 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.265613 May 23 15:50:37.190508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 414/765] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:37.195274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:37.195733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:37.196505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:37.196947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:37.361303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:37.369663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5c263d0e-5889-4fff-83b5-051c305ffc17 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.184270 May 23 15:50:37.370741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 352/766] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:38.200301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:38.200902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:38.207873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:38.208695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:38.211837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.211837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.211837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.211837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.216046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.216499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.217274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.217710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.414259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:38.417608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:38.422879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:38.424500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:38.424885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b680c169-a4ac-4769-83ec-e4ad8ca94dad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.228428 May 23 15:50:38.425450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 415/767] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:38.425667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-641945cf-77d0-47d7-a1e0-cd61bf90c4a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.221870 May 23 15:50:38.426407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 353/768] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:38.430917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:38.431561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:38.437408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.437859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.438615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:38.439055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:38.567358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:38.574775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e93a91a9-21c3-4c1f-9346-019d482dc767 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148328 May 23 15:50:38.575365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 416/769] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:39.441984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:39.442508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:39.443232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:39.443838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:39.448760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.449213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.449972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.450316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.450431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.450765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.451546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.451974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.664790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:39.672127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:39.673373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-42dcd4b9-9735-4ba1-8875-bffeab822b96 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.235121 May 23 15:50:39.673952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 354/770] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:39.679642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:39.687814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:39.688011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:39.688011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:39.688011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bae52253-4c03-40db-a354-e078c9e37db4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.245166 May 23 15:50:39.688340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 417/771] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:39.688435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.688757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.689537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:39.689975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:39.819923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:39.827244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d986f385-7b6f-4585-a3e3-55aeeb20dda1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.152318 May 23 15:50:39.827801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 355/772] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:40.699180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:40.699804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:40.703738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:40.704378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:40.706667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.707134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:40.707913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.708350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:40.710736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.711214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:40.713850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.713850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:40.901415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:40.905596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:40.908939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2153964b-062d-4292-b266-2502f281d664 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.213582 May 23 15:50:40.909539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 418/773] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:40.914870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:40.915606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:40.918508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:40.920039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:40.921199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cfe40779-b6c7-41f5-86de-6802cfdbc92c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.221352 May 23 15:50:40.921306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.921707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:40.921815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 356/774] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:40.922471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:40.922976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:41.059624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:41.067074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cdff4e9a-a1de-4b84-91ff-4ca33b9ac346 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.156378 May 23 15:50:41.067679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 419/775] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:40 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:41.927382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:41.930019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:41.939948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:41.941953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:41.942374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=53 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:41.947712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:41.952846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:41.952846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:41.952846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:41.953416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:41.953416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:41.953416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:41.953416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:42.245979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:42.247593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:42.255749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe5e5aff-c3a7-42e2-b4f2-8a5a2143c53b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.333336 May 23 15:50:42.257221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:42.259064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:42.260359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-517cc444-29a3-45a9-974d-bf12733c150a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.324503 May 23 15:50:42.260995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 420/776] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:42.265286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 357/777] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:41 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:42.265875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:42.266709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:42.279477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:42.279934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:42.280719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:42.281692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:42.353452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-c4eb5cf0-24a6-426a-b9a9-92693a06a99f req-3ac6c2ff-bdf4-4e84-97fc-c2672d5ea5df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a14c169-ee02-4abd-8bae-be450c73183c", "tag": "c91045a8-7f3a-40f1-bb8a-d0e2283349f5"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:42.363214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-c4eb5cf0-24a6-426a-b9a9-92693a06a99f req-3ac6c2ff-bdf4-4e84-97fc-c2672d5ea5df service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:42.363874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-c4eb5cf0-24a6-426a-b9a9-92693a06a99f req-3ac6c2ff-bdf4-4e84-97fc-c2672d5ea5df service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:42.386588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-c4eb5cf0-24a6-426a-b9a9-92693a06a99f req-3ac6c2ff-bdf4-4e84-97fc-c2672d5ea5df service nova] Creating event network-changed:c91045a8-7f3a-40f1-bb8a-d0e2283349f5 for instance 3a14c169-ee02-4abd-8bae-be450c73183c on hv-cinder-83588 May 23 15:50:42.391828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-c4eb5cf0-24a6-426a-b9a9-92693a06a99f req-3ac6c2ff-bdf4-4e84-97fc-c2672d5ea5df service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.125309 May 23 15:50:42.392408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 358/778] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:50:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:42.441101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:42.449277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-192bcf48-8f05-48c9-af66-9b9e57a43846 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.187021 May 23 15:50:42.449838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 421/779] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:42.452016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-86d32634-8dd4-4e8b-b5d1-d99a5c6629ef req-61d28a26-666f-4a11-8bbc-3dd353d4728c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:42.460853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-86d32634-8dd4-4e8b-b5d1-d99a5c6629ef req-61d28a26-666f-4a11-8bbc-3dd353d4728c service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:42.461296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-86d32634-8dd4-4e8b-b5d1-d99a5c6629ef req-61d28a26-666f-4a11-8bbc-3dd353d4728c service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:42.471494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-86d32634-8dd4-4e8b-b5d1-d99a5c6629ef req-61d28a26-666f-4a11-8bbc-3dd353d4728c service nova] Creating event network-changed:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 15:50:42.476398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-86d32634-8dd4-4e8b-b5d1-d99a5c6629ef req-61d28a26-666f-4a11-8bbc-3dd353d4728c service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082596 May 23 15:50:42.477128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 359/780] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:50:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:43.272783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:43.276464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:43.280214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.280659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.281448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.282028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:43.282450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.282864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:43.290199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.290651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.291553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.292020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.478159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:43.486440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-15461e6f-a5cf-43c7-812e-81b247e24993 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.217637 May 23 15:50:43.487030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 422/781] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:43.492178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:43.492392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:43.492857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:43.499199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.499651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.500688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:43.501256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:43.502301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:43.503933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:43.505116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-18877726-cc91-4b79-881e-c7c216334f90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.227801 May 23 15:50:43.505677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 360/782] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:43.521496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:43.645885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:43.653527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-be1dd422-51b5-48f3-b625-fc28d3456c62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.165472 May 23 15:50:43.654102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 423/783] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:44.504262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:44.505565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:44.512312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.512761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.513552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.513986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.522501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:44.523233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:44.529660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.530182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.531017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.531660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.691358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:44.698855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a5eb7a2-b397-49e1-b8ed-4de379f47356 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.198624 May 23 15:50:44.699536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 361/784] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:44.704133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:44.704749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:44.710924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.711419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.712188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:44.712625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:44.717315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:44.726311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:44.727988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:44.729239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9fc40387-34e8-4a3d-a38c-abcc6a52b567 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.210625 May 23 15:50:44.729885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 424/785] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:44.928515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:44.936996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a396fa80-fe9f-4649-9fc0-4244fe384d95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.236408 May 23 15:50:44.937632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 362/786] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:45.643090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f87f5985-6972-4efb-8acc-b66fe727ea3d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:45.650912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f87f5985-6972-4efb-8acc-b66fe727ea3d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.651388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f87f5985-6972-4efb-8acc-b66fe727ea3d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.713385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f87f5985-6972-4efb-8acc-b66fe727ea3d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Going to try to stop instance {{(pid=124966) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 23 15:50:45.715957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:45.717131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:45.723751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.724258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.725213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.725649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.860718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:45.868806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b8a6b92c-651a-4f76-ba8b-ea2b8c23a06c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.156647 May 23 15:50:45.869374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 363/787] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:45.873780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:45.874376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:45.881071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.888585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.888585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.888585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.888585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:45.913005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f87f5985-6972-4efb-8acc-b66fe727ea3d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action" status: 202 len: 0 microversion: 2.1 time: 0.273551 May 23 15:50:45.913605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 425/788] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:50:45 2022] POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:50:45.922424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:45.923140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:45.931069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.931554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:45.932326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:45.932763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.070052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:46.078816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:46.080427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:46.081622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5c58fe24-2c0e-49ef-8ce7-d1d9e19a2468 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.211326 May 23 15:50:46.083212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 364/789] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:46.087305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:46.087923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:46.093840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.095387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.095387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.095639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.099421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:46.107540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4a8ce93f-9b50-4462-b10e-71906dfc718c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.189294 May 23 15:50:46.108293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 426/790] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:46.224638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:46.232126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-766ffde5-8e81-4f11-a362-e317f7af0a88 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148309 May 23 15:50:46.232745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 365/791] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:46.389594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8ebaf0a3-1be2-4045-b12b-f4ad1873a26a"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:46.390751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:46.397480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.397927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.398685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.399141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.886528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:46.887173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:46.893784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.894238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:46.894999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:46.895478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.024602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.031954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-22b6a416-7117-4d36-9323-9ac19c34ab2b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.149294 May 23 15:50:47.032508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 366/792] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:46 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:47.101600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:47.102208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:47.108154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.108600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.109348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.109776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.237053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.246838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.248362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.249598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-32fc43bd-6d3f-4a5f-bf7e-2f25d0109487 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.152502 May 23 15:50:47.250409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 367/793] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:47.254745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:47.269779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:47.269779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.269779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.269779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.270658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.389359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.397772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c783a63c-23ba-42a6-8ae9-c68ce91b9f44 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.146427 May 23 15:50:47.398333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 368/794] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:47.402633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:47.403267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:47.408896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.409347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.410115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.410549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.541931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:47.549518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b315b323-d01a-4a1f-ab15-1c01f191e3f1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.150216 May 23 15:50:47.550084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 369/795] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:47.632877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-c6c207b1-b35c-4bb8-b714-5b70e1f42dbd req-403c05f0-d5c7-4736-8944-af37cf0c6615 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "name": "network-vif-plugged", "status": "completed", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:47.641069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-c6c207b1-b35c-4bb8-b714-5b70e1f42dbd req-403c05f0-d5c7-4736-8944-af37cf0c6615 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:47.641600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-c6c207b1-b35c-4bb8-b714-5b70e1f42dbd req-403c05f0-d5c7-4736-8944-af37cf0c6615 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:47.653053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-c6c207b1-b35c-4bb8-b714-5b70e1f42dbd req-403c05f0-d5c7-4736-8944-af37cf0c6615 service nova] Creating event network-vif-plugged:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 15:50:47.658203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-c6c207b1-b35c-4bb8-b714-5b70e1f42dbd req-403c05f0-d5c7-4736-8944-af37cf0c6615 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082902 May 23 15:50:47.658804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 370/796] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:50:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:50:48.049714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:48.050417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:48.057143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.057592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.058347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.058781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.187410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.195258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e57b5349-a510-404b-8d6f-869539db49f0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.150271 May 23 15:50:48.195824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 371/797] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:48 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:48.276422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:48.277051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:48.283002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.283477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.284230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.284661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.413095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.421453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.422946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.424130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fc2ed3f1-8d6c-47cd-85df-2b6d6c9a230f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.151574 May 23 15:50:48.424688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 372/798] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:48.429588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:48.430198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:48.435929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.436377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.437139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.437573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.564460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.573452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7f293bd8-44cf-4019-bf05-6b4e398e63f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.147775 May 23 15:50:48.574030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 373/799] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:48.578326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:48.578922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:48.584831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.585275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.586020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:48.586447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:48.715061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:48.722321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-65526a9f-ac58-4f76-9088-e542b2c24905 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.147346 May 23 15:50:48.722866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 374/800] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:49.212512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:49.213148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:49.219545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.220005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.220744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.221172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.347854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:49.355383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36964580-f307-4959-bf82-bb5a843cec4e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.147330 May 23 15:50:49.355949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 375/801] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:49.442873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:49.443620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:49.449598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.450044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.450826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.451290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.584918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:49.593404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:49.595003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:49.596200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-981842b0-a749-4b54-9a82-17aed0147ccc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.157862 May 23 15:50:49.596757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 376/802] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:49.601245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:49.601859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:49.607933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.608390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.609272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.609713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.977473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:49.985552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-faf8dd40-ebbb-4c55-bf94-f197388be6a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.387856 May 23 15:50:49.986113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 377/803] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:49.991087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:49.991723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:49.997388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.997829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:49.998577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:49.999030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:50.127177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:50.134595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-34d8c7cc-9182-4c4a-8547-9e33018bbfe1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.147559 May 23 15:50:50.135169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 378/804] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:50.372368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:50.373035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:50.379903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:50.380356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:50.381118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:50.381590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:50.511544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:50.518998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9548dcae-c208-4d6a-a5b6-93200409fda4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.150450 May 23 15:50:50.519609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 379/805] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:50.614458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:50.615127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:50.621034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:50.626124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:50.626124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:50.626124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:50.757855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:50.766659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:50.768118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:50.769307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7c9b9331-3004-49c7-bc44-d503280633e1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.158781 May 23 15:50:50.769867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 380/806] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:51.003710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:51.004338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:51.010943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.011432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.012191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.012622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.030428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:51.140832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.149236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1cab131-4dcc-469a-8bc7-89097471d9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150146 May 23 15:50:51.149792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 381/807] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:51.154073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:51.154677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:51.160267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.160715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.162117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.162555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.293521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.301168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c3888c00-6684-42d9-b2af-e1e2b2116df6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.150439 May 23 15:50:51.301738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 382/808] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:51 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:51.536700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:51.537325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:51.543998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.544394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.545165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.545603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.670769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.678200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abac40c6-bd4b-4ebf-a173-d26a00535230 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.145463 May 23 15:50:51.678764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 383/809] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:51.788011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:51.788630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:51.794711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.795128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.795900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:51.796345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:51.926457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.935346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.936844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:51.937994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f4d7858-46d5-42ea-b7d0-7bdce5bfa78b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.153932 May 23 15:50:51.938551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 384/810] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:52.167022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:52.167660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:52.173875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.174316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.175078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.175532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.464486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:52.472522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3207a013-134b-4f5b-a54f-5ecc80743a15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.309469 May 23 15:50:52.473103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 385/811] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:52.477978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-781dbbea-5d51-4861-9bdb-56cd34f47d81 req-54fd6e03-b423-42c1-ba0e-b7c108ebc282 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "name": "network-vif-plugged", "status": "completed", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:52.485694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-781dbbea-5d51-4861-9bdb-56cd34f47d81 req-54fd6e03-b423-42c1-ba0e-b7c108ebc282 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.486140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-781dbbea-5d51-4861-9bdb-56cd34f47d81 req-54fd6e03-b423-42c1-ba0e-b7c108ebc282 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.497228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-781dbbea-5d51-4861-9bdb-56cd34f47d81 req-54fd6e03-b423-42c1-ba0e-b7c108ebc282 service nova] Creating event network-vif-plugged:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 15:50:52.502340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-781dbbea-5d51-4861-9bdb-56cd34f47d81 req-54fd6e03-b423-42c1-ba0e-b7c108ebc282 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028163 May 23 15:50:52.502899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 386/812] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:50:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:52.508387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:52.509083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:52.515235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.515683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.516435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.516869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.644938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:52.652863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cbed2422-e7f9-4113-af6c-f49f4c965463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148870 May 23 15:50:52.653415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 387/813] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:52.694820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:52.695454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:52.701254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.701697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.702442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.702874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.832848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:52.840822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d9a259f-76a0-4556-9789-21d1fd825ad5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.149691 May 23 15:50:52.841375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 388/814] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:52.956053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:52.956577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:52.963128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.963581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:52.964331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:52.964763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.094207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:53.103515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:53.105192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:53.106367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6f781af-dedb-4e83-ab90-f1850a36d910 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.154181 May 23 15:50:53.106914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 389/815] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:53.491073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:53.491626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:53.497736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.498189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.498955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.499434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.629819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:53.637690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a857b382-8277-4cd3-b1e9-47028237d869 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.151307 May 23 15:50:53.638243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 390/816] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:53.669359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:53.670003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:53.675895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.676338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.677622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.678052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.809181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:53.816579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83cb0806-22f1-45b4-9b94-6d4d1e941e3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.150854 May 23 15:50:53.817124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 391/817] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:53.857630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:53.858240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:53.864167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.865198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.865940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:53.866373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:53.997442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.005378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-638f9943-9a81-4281-9350-da0ad192f9c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.151214 May 23 15:50:54.005873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 392/818] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:54.126653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:54.126895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:54.134014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.134461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.135300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.135726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.274201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.282784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.284328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.285545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f80574e9-7d73-4a29-92d8-364b1ab918e5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.163727 May 23 15:50:54.286089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 393/819] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:54.576861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7db593a-91d9-4072-885a-0f45a332d05e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 8.286107 May 23 15:50:54.577404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 427/820] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:50:46 2022] POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 194 bytes in 8288 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:54.654850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:54.655510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:54.662021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.662475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.663285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.663734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.797575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.806027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-750793a2-c4ca-4f72-bcc2-ad79b4fd2b6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.155011 May 23 15:50:54.810141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 394/821] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:54.833947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:54.834586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:54.841087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.841543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.842363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:54.842806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:54.970078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:54.977643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bf2b23d-e301-4422-a999-5ef7854cf5d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148268 May 23 15:50:54.978263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 428/822] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:54 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:55.021744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:55.022361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:55.028439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.033393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.033393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.033393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.208999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:55.216520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eafaf54c-cd63-426c-91e3-cef7016e921c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.198730 May 23 15:50:55.217078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 395/823] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:55.304972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:55.305587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:55.312011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.312460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.313216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.313652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.449125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:55.457914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:55.459521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:55.460688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2beffd3c-c86e-414a-85e9-bf040689606a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.160332 May 23 15:50:55.461254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 429/824] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:55.823058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:55.823697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:55.829770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.830221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.830979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:55.831446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:55.961723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:55.969739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4bfefa68-7cdb-4758-a154-cae9b095fe0c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150560 May 23 15:50:55.970298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 396/825] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:56.004037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:56.004683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:56.011447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.011899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.012726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.013184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.140687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:56.148521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4c0170eb-011f-44dd-93b2-670f05ef0c50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148557 May 23 15:50:56.149073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 430/826] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:56.234101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:56.234869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:56.242807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.243319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.244148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.244618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.376605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:56.384778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b61301b3-7858-4230-9cb1-e8e01875ec38 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.154878 May 23 15:50:56.385338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 397/827] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:56.478842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:56.479477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:56.485992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.486383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.487160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.487597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.617449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:56.626005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:56.627539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:56.628730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-311d64e3-75ce-4729-b315-c6dcc664a8e7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.153829 May 23 15:50:56.629290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 431/828] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:56.986968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:56.987712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:56.994622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.995076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:56.995882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:56.996319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.125419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.129012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:57.133794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b6133a6-5c40-40bd-96c1-a243b7662b30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150783 May 23 15:50:57.137674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 398/829] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:57.165890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:57.166506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:57.172726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.173173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.173940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.174368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.210371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-2b820dda-4d9f-47e6-866f-636a1f87568e req-c7c2f4b8-faa2-470e-9c79-97276e9d13b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a14c169-ee02-4abd-8bae-be450c73183c", "name": "network-vif-plugged", "status": "completed", "tag": "c91045a8-7f3a-40f1-bb8a-d0e2283349f5"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:50:57.219114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2b820dda-4d9f-47e6-866f-636a1f87568e req-c7c2f4b8-faa2-470e-9c79-97276e9d13b7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.219565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2b820dda-4d9f-47e6-866f-636a1f87568e req-c7c2f4b8-faa2-470e-9c79-97276e9d13b7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.232867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-2b820dda-4d9f-47e6-866f-636a1f87568e req-c7c2f4b8-faa2-470e-9c79-97276e9d13b7 service nova] Creating event network-vif-plugged:c91045a8-7f3a-40f1-bb8a-d0e2283349f5 for instance 3a14c169-ee02-4abd-8bae-be450c73183c on hv-cinder-83588 May 23 15:50:57.240083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-2b820dda-4d9f-47e6-866f-636a1f87568e req-c7c2f4b8-faa2-470e-9c79-97276e9d13b7 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033766 May 23 15:50:57.246720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 399/830] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:50:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:50:57.402425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.402425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45545a11-64d0-44f6-87b6-595127f8dd1e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.160681 May 23 15:50:57.402425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 432/831] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:57.402988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:57.403228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:57.411497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.412282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.413250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.414147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.563183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.571259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17143b4e-5f2c-4f82-9265-f455774908fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1358 microversion: 2.1 time: 0.173329 May 23 15:50:57.571818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 400/832] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:57.647193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:57.647828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:57.654195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.654642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.655446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:57.655882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:57.786848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.795156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.796770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:57.797956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-003f0cce-36f9-4970-8900-2df235d148af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.155167 May 23 15:50:57.798501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 433/833] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:50:58.152143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.153457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:58.159969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.160422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.162078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.162199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.292264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:58.300214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6adafb4a-f41c-41b9-8994-07d40c52946e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.152276 May 23 15:50:58.300887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 401/834] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:58.343215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.343999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:58.350515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.350957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.351777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.352215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.483471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:58.491485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02d1b9cd-6224-4b27-b29f-0d06b7fc2d36 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.152166 May 23 15:50:58.492043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 434/835] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:58.588595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.589208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:58.595230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.595683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.596438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.596866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.724911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:58.732319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25d3a9a5-8dfb-4d54-9274-437c25f8eac8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1549 microversion: 2.1 time: 0.147602 May 23 15:50:58.732881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 402/836] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1549 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:58.748195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.748806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:58.755457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.755896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.756643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.757074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.827668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.828295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:58.834834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.835331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.836137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.836596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.934021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:58.942026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-09a90741-8760-43a2-9b43-93ada1b975c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1549 microversion: 2.1 time: 0.197503 May 23 15:50:58.943196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 435/837] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1549 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:58.958188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:58.962400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:58.963226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Searching by: {'deleted': False, 'project_id': '8ce8b36a28714d618becacd8e2925a75'} {{(pid=124966) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 23 15:50:58.969772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.970222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.971404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:58.971534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:58.990193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.011195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.013170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.014212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ff8f609-05e9-46af-8ee1-9c10476c8541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.190397 May 23 15:50:59.015185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 403/838] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:50:59.087184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.multi_cell_list [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=124966) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 23 15:50:59.173446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4314e3f4-c35e-4c38-a4bb-de3649c928d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.219145 May 23 15:50:59.174032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 436/839] 10.222.0.32 () {58 vars in 1124 bytes} [Mon May 23 15:50:58 2022] GET /compute/v2.1/servers => generated 342 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:50:59.184478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:59.185090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:59.191216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.191663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.192974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.193410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.318521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:59.319257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:59.321105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.325622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.326068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.326833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.327299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.328839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-01a79acb-725e-413c-b4fe-17c60e47aabe tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1549 microversion: 2.1 time: 0.148474 May 23 15:50:59.329400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 404/840] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1549 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:50:59.348920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:59.466134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.473782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32f6396f-fe1f-4cb8-980a-7d6d39488e55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.159890 May 23 15:50:59.474348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 437/841] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:50:59.508999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:50:59.509539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:50:59.515604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.516051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.516814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:50:59.517252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:50:59.649826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:50:59.657258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7999f626-8d99-44c5-b2af-ff88516fdd69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.152826 May 23 15:50:59.657854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 405/842] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:50:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:00.037961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:00.038581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:00.045415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.048264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.048264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.048264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.200774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:00.210602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:00.214887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:00.214887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-faef9003-0be7-4c00-b34b-4e0146527528 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.180234 May 23 15:51:00.214887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 438/843] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:51:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:51:00.489853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:00.490466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:00.497010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.497460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.498212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.498644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.625957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:00.634619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0fe7e5a0-fa4a-44b4-9bd3-689b26fdc408 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.148633 May 23 15:51:00.635203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 406/844] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:00.673690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:00.674303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:00.680422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.683392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.683392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:00.683392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:00.808162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:00.815490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e34a6bb1-14c8-448e-886e-2ac75a9bdaa7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.145635 May 23 15:51:00.816057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 439/845] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:00 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:01.232187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:01.232875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:01.235047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:01.240734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.241316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:01.242099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.242556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:01.375021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:01.383831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:01.387906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:01.387906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-29a495f7-529e-4782-9763-5ebdaf6590db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.158305 May 23 15:51:01.387906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 407/846] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:51:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 15:51:01.652268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:01.652829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:01.659791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.660236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:01.660987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.662051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:01.792031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:01.799979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8be2fd48-3e95-44fd-bbe4-c9144ae4f974 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.151601 May 23 15:51:01.800545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 440/847] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:01.832773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:01.833988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:01.840461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.840907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:01.841662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:01.842093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.063183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:02.073180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fb33289e-f60d-43a3-b3df-19c50280603c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.244236 May 23 15:51:02.073931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 408/848] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:02.274892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "15326cd2-64ce-47d8-90db-d8ae702d0251"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:02.276086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:02.282095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.282549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.283349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.283791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.404757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:02.405362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:02.411354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.411802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.412555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.413016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.552566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:02.560872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:02.562357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:02.563549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8071935c-67ff-4ddb-9cdc-5b94c2b041df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1806 microversion: 2.20 time: 0.162743 May 23 15:51:02.568152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 409/849] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:51:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1806 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:51:02.816945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:02.818016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:02.824001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.824692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.833989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:02.834622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:02.964127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:02.972278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b12a0407-5864-4571-8f55-97467ed806b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.159220 May 23 15:51:02.972853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 410/850] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:03.089965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:03.090490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:03.096485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.100780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:03.100780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.100780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:03.241674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:03.249225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b872162-1512-4b45-bb18-4d9884cde5a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.163387 May 23 15:51:03.269820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 411/851] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:03.581966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:03.582586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:03.589447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.591625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:03.591625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.591625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:03.719933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:03.725015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-53e2e9fb-e5e6-49bb-8ff6-87bd485e8b9e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.453881 May 23 15:51:03.725631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 441/852] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:51:02 2022] POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments => generated 194 bytes in 1455 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:03.728499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:03.730030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:03.731275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a40ddae-9bbc-42c5-8e7b-4c66c2e1db0a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1793 microversion: 2.20 time: 0.153308 May 23 15:51:03.731823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 412/853] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 15:51:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1793 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 15:51:03.989093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:03.994601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:03.996774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.997223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:03.997988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:03.998426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:04.127665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:04.135715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c821ed8a-d70e-49b6-98f9-b9850e8d318e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150959 May 23 15:51:04.136266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 442/854] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:04.273149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:04.273761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:04.280244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:04.280631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:04.281380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:04.281813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:04.409867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:04.417692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e59d393c-d243-484e-ab05-50d2492886ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.148372 May 23 15:51:04.418252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 413/855] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:05.152853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:05.153418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:05.159824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:05.160263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:05.161008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:05.161946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:05.307023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:05.315116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76ce8007-a05e-49b2-b67d-e5a48791f28f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.166156 May 23 15:51:05.316777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 443/856] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:05.434125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:05.434688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:05.441242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:05.441692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:05.442452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:05.443450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:05.574382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:05.581739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95c24700-b5b9-4f59-9de6-b0f9206a0998 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.151584 May 23 15:51:05.582307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 414/857] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:06.331970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:06.332584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:06.335007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=62 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:06.339828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:06.340279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:06.341038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:06.341474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:06.597698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:06.598340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:06.605419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:06.605867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:06.606630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:06.607066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:06.628689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:06.636547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-192073bd-7caf-4a0d-a634-e45f1fe8bf7c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.308463 May 23 15:51:06.637113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 444/858] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:06.732733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:06.740615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-44f45e13-9352-4dc2-9e4a-d0d6cf342df2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1318 microversion: 2.1 time: 0.146783 May 23 15:51:06.741178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 415/859] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1318 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:07.653498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:07.654116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:07.660589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.661039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:07.662368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.662807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:07.757892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:07.758603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:07.765170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.765706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:07.766556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.767085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:07.771261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:07.791969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:07.799867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89b873ba-3f1c-4417-8d3f-2328ce648b2d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150288 May 23 15:51:07.800431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 445/860] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:07.897392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:07.907758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb6a8995-6fa8-44aa-a953-449a49ef63dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1506 microversion: 2.1 time: 0.151831 May 23 15:51:07.907758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 416/861] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1506 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:07.924695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "58902dfa-8f68-4cae-a849-ef582736a197", "device": "/dev/sdb"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:07.926306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:07.932904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.936176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:07.936176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:07.936176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:08.816569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:08.817184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:08.823823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:08.833075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:08.833075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:08.833075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:08.968041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:08.976603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-894d5db4-a271-4304-b06c-a72e83339230 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.163897 May 23 15:51:08.977166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 417/862] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:09.001753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:09.462553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d90dfc44-1108-4ed0-b95d-0a7336e6280c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "POST /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.543135 May 23 15:51:09.463539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 446/863] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:51:07 2022] POST /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments => generated 194 bytes in 1545 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:09.702775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:09.708022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:09.709977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.710434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.711272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.711714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.734737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1343c76c-7540-476d-87b1-047f4674abff tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/ips" status: 200 len: 149 microversion: 2.1 time: 0.135688 May 23 15:51:09.735481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 418/864] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 15:51:09 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/ips => generated 149 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:09.746974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e1263857-63cf-45c6-8a18-ae4614c86f1d"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:09.748271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:09.754806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.755852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.756605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.757037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.844769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: POST /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/os-volume_attachments HTTP/1.1 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 74 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:51:09.846517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {"volumeAttachment": {"volumeId": "e1263857-63cf-45c6-8a18-ae4614c86f1d"}} May 23 15:51:09.848216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': 'c7915105-453d-453a-ad3c-656139c27d5a', 'body': {'volumeAttachment': {'volumeId': 'e1263857-63cf-45c6-8a18-ae4614c86f1d'}}} May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:09.849174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:09.850795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 400 to user: Invalid volume: volume e1263857-63cf-45c6-8a18-ae4614c86f1d is already attached to instances: 4c5a0e23-b645-46dd-8924-37f041a76f7c {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32b893f2-5c94-4e1c-a64c-035080506170 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.105592 May 23 15:51:09.852804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 447/865] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 15:51:09 2022] POST /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a/os-volume_attachments => generated 174 bytes in 107 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:51:09.857188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:09.857676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:09.864046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.865028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.865813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:09.866258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:09.891431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 15:51:09.995588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:09.999068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:10.003425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.004015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.004801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.005855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.135059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-81461baa-9882-4567-ac47-a01d7592f126 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 204 len: 0 microversion: 2.1 time: 0.281539 May 23 15:51:10.135793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 419/866] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 15:51:09 2022] DELETE /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 15:51:10.142681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:10.144765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:10.145502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:10.147822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-143d105c-fc6e-458d-abc3-b305e1392491 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.157080 May 23 15:51:10.148398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 448/867] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:10.152004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.152457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.153254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.153690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.298120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:10.301389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3c8fffaf-fe31-4c54-9804-646d0de1d07a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1682 microversion: 2.1 time: 0.160783 May 23 15:51:10.302617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 420/868] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:10 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:10.588278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:10.589453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:10.595774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.596920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.597690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.598122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.629315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'flavor' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:51:10.793454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=6,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:10.964748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0975264b-cca8-41ca-9fa2-b74de78f3c86 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action" status: 200 len: 32 microversion: 2.1 time: 0.380607 May 23 15:51:10.965272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 449/869] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:51:10 2022] POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action => generated 32 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 15:51:10.974717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:10.975357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:10.981723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.982173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:10.982938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:10.983408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:11.113630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:11.122289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68187243-460d-4494-9412-a861e1023a69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151905 May 23 15:51:11.122855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 421/870] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:11.166391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:11.167190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:11.173443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:11.173891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:11.174654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:11.175087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:11.303342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:11.311426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3a746431-b5a3-459a-9161-452d581a76b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.150287 May 23 15:51:11.311995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 450/871] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:11.321861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:11.322476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:11.328539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:11.328941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:11.329704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:11.330155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:11.464005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:11.471463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-90fcafd4-7178-4262-abc0-a29bec7a7d59 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1682 microversion: 2.1 time: 0.153920 May 23 15:51:11.472031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 422/872] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:11 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1682 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:12.139829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:12.140459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:12.147728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.148168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.148921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.149353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.329223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:12.329940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:12.336514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.337052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.337821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.338258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.362287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:12.371371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-038bb414-1f7b-43f9-bacf-9dad4e7d2c28 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.235421 May 23 15:51:12.371936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 451/873] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:12.488653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:12.489261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:12.495820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.496268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.497022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:12.497451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:12.526678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:12.534959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4c344132-861b-4e85-944c-063d0ff07f34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.209601 May 23 15:51:12.535648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 423/874] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:12.660376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:12.668073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce0e1cfb-f86d-4554-b396-78cb40d1c0ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1682 microversion: 2.1 time: 0.183341 May 23 15:51:12.668681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 452/875] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:12 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1682 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:13.388734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:13.389565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:13.397567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.397567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.398038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.398472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.540865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:13.550748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-63f3b1f8-089c-4ddc-9a71-00c6dbd01aaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.165882 May 23 15:51:13.554529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:13.556069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:13.561663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 424/876] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:13.564078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.565252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.566530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.566817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.684387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:13.685302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:13.691878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.692393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.693147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:13.693591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:13.710887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:13.716354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-604ed1fe-a056-46e6-9993-18470204fe99 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.167149 May 23 15:51:13.717143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 453/877] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:13.834281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:13.841807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-683b79e7-d3ed-4672-93c2-b56e7b3f34b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1682 microversion: 2.1 time: 0.161487 May 23 15:51:13.842369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 425/878] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:13 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:14.212428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-6c9b43ed-601a-4804-b950-fab8680443e7 req-c26cd0fb-4f77-4e87-88f9-2e892922b554 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7915105-453d-453a-ad3c-656139c27d5a", "name": "network-vif-deleted", "tag": "5f5a9b4b-5b24-4108-88fc-c41edee8bf44"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:14.224406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-6c9b43ed-601a-4804-b950-fab8680443e7 req-c26cd0fb-4f77-4e87-88f9-2e892922b554 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.224998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-6c9b43ed-601a-4804-b950-fab8680443e7 req-c26cd0fb-4f77-4e87-88f9-2e892922b554 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.243354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-6c9b43ed-601a-4804-b950-fab8680443e7 req-c26cd0fb-4f77-4e87-88f9-2e892922b554 service nova] Creating event network-vif-deleted:5f5a9b4b-5b24-4108-88fc-c41edee8bf44 for instance c7915105-453d-453a-ad3c-656139c27d5a on hv-cinder-83588 May 23 15:51:14.248686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-6c9b43ed-601a-4804-b950-fab8680443e7 req-c26cd0fb-4f77-4e87-88f9-2e892922b554 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040581 May 23 15:51:14.249860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 454/879] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:51:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:14.568786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:14.569720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:14.576620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.576858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.577643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.578512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.732608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:14.734739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:14.735561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:14.743040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5153d06d-65e2-4996-bf8a-b4b8822f39b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.179116 May 23 15:51:14.751150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 426/880] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:14.752685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.753326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.754151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.754597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.862091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:14.862871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:14.873106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.873615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.874521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.875828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.904485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:14.913671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d78982f1-f2e7-48c2-90b2-774c06e2546a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.183230 May 23 15:51:14.923209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 455/881] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:14.923334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:14.923334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:14.923334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lazy-loading 'fault' on Instance uuid c7915105-453d-453a-ad3c-656139c27d5a {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:51:14.963427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:14.972018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08b9df79-2363-41cb-96bd-40db204a7673 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 200 len: 1332 microversion: 2.1 time: 0.116077 May 23 15:51:14.978539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 427/882] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:14 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 1332 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:15.761011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:15.761703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:15.768639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:15.769100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:15.769871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:15.770312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:15.932581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:15.934200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:15.945836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:15.946004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:15.946004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:15.946653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:15.947128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:15.954284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b780c04f-ad1d-416e-a0b3-e630666ac07f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.197124 May 23 15:51:15.954855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 456/883] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:16.003497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:16.004443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:16.012458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:16.013121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:16.014548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:16.014548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:16.030537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a HTTP/1.1 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:51:16.032871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 15:51:16.040826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'c7915105-453d-453a-ad3c-656139c27d5a'} May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:51:16.041706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 15:51:16.043297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:16.044598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:51:16.044598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0fe1a0ea-adc3-49e9-8519-eceb342c6fcc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 404 len: 111 microversion: 2.1 time: 0.035927 May 23 15:51:16.044598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 457/884] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:15 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 15:51:16.050171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2f34e91-a972-4b12-a274-5527e6d9151b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '.version_select at 0x7f47f8f92b80>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:16.134167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:16.140106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2f34e91-a972-4b12-a274-5527e6d9151b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1190510603" status: 202 len: 0 microversion: 2.1 time: 0.093746 May 23 15:51:16.141208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 458/885] 10.222.0.32 () {58 vars in 1220 bytes} [Mon May 23 15:51:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1190510603 => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:51:16.147670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7a40b2a-dd38-4f15-b466-c7a9cff2b969 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.217938 May 23 15:51:16.159613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 428/886] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:16.977527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:16.978146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:16.985010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:16.985461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:16.986234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:16.986678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.070648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:17.071172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:17.078392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.078841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.079646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.080084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.120940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:17.129178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e5ec5b8-178d-45db-af5e-b57948f2e3cf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.156126 May 23 15:51:17.129735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 459/887] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:17.167464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:17.168015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:17.169979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lazy-loading 'services' on Instance uuid 4c5a0e23-b645-46dd-8924-37f041a76f7c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:51:17.174349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.174793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.175584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.176017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.326471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:17.333212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3541d2f1-7ce4-4b43-9ce5-51ad31b0e76b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.170140 May 23 15:51:17.333915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 460/888] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:17.420710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2cb923ee-d7b3-43c2-9e37-bbc86ef19470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments/e1263857-63cf-45c6-8a18-ae4614c86f1d" status: 202 len: 0 microversion: 2.1 time: 0.354444 May 23 15:51:17.421225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 429/889] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:51:17 2022] DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments/e1263857-63cf-45c6-8a18-ae4614c86f1d => generated 0 bytes in 356 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 15:51:17.430631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:17.431601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:17.438131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.438578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.439442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:17.439876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:17.470452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8027220b-7c27-4548-82e8-58aad140dda9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044588 May 23 15:51:17.470996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 461/890] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:18.146714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:18.147631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:18.154741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.159687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.159687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.159687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.176325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:18.295318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:18.303990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fbb7ed9b-7b5a-4dd6-83c6-ce7f2d633eec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.162085 May 23 15:51:18.304546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 430/891] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:18.350625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:18.351261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:18.359840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.360584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.361538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.361981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.482043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:18.483036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:18.489051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.490086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.490881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:18.491350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:18.492943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:18.501268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d2514b0-59e1-44ac-beb9-66165e4d17ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.154887 May 23 15:51:18.501985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 462/892] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:18.520784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-69ae912d-2083-4db9-bee9-b4983347d787 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042756 May 23 15:51:18.521332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 431/893] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:19.321522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:19.322135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:19.328395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.328841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.329595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.330026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.463249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:19.471367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-545a20cb-ee1a-436e-931b-221990f59fae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.154050 May 23 15:51:19.471929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 463/894] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:19.517706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:19.518415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:19.525615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.526070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.526834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.527303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.532327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:19.533491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:19.540006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.540453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.541212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.541645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.571130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f474353-7607-4010-8d70-30ab6d392447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042505 May 23 15:51:19.571717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 464/895] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:19.785782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:19.786448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:19.792475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.792923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.793690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:19.794130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:19.941168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:19.949386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-14bc906c-5500-49ce-89fd-19c39ba9e85b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.435453 May 23 15:51:19.949925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 432/896] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:19.961931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:19.971131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a16f5553-04af-4379-a132-7bc233ed01e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1595 microversion: 2.1 time: 0.188485 May 23 15:51:19.976196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 465/897] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1595 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:20.489011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:20.489722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:20.495987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.496440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:20.497222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.497652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:20.582502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:20.584172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:20.590884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.591731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:20.592751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.593489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:20.630590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e73c5905-6216-4f19-8ae5-76b1dd2ef344 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051739 May 23 15:51:20.632292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 466/898] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:20.635278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:20.644456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61ac0b17-f256-45a5-a34a-0175cfdfefaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.159293 May 23 15:51:20.645042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 433/899] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:20.966589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:20.967301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:20.973400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.973899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:20.974678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:20.975140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:21.106849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:21.115454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5f0384e2-86b0-4c32-854e-ffa0d3183bca tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.152852 May 23 15:51:21.116036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 467/900] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:21.642620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:21.643905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:21.651518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:21.652129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:21.652917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:21.653356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:21.662539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:21.663476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:21.670127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:21.670713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:21.671789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:21.673570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:21.687951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0b2d77a1-8738-4894-9359-67905523742c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049181 May 23 15:51:21.692219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 434/901] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:21.697148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:21.826544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:21.835694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ea7f8258-a9c7-4d2b-be75-173b6d1b7441 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.177715 May 23 15:51:21.836275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 468/902] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:22.132710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:22.133329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:22.139657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.140099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.141401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.141573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.219821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-2e2b647a-07f2-4e71-8b26-63c7924a7418 req-301ba07b-9f44-4a19-baac-9aaad389849c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:22.239924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-2e2b647a-07f2-4e71-8b26-63c7924a7418 req-301ba07b-9f44-4a19-baac-9aaad389849c service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.240344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-2e2b647a-07f2-4e71-8b26-63c7924a7418 req-301ba07b-9f44-4a19-baac-9aaad389849c service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.259155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-2e2b647a-07f2-4e71-8b26-63c7924a7418 req-301ba07b-9f44-4a19-baac-9aaad389849c service nova] Creating event network-changed:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 15:51:22.264654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-2e2b647a-07f2-4e71-8b26-63c7924a7418 req-301ba07b-9f44-4a19-baac-9aaad389849c service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049660 May 23 15:51:22.265524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 469/903] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:51:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:22.302933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:22.313774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a1196cf4-4483-44c6-8355-787596792a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.182722 May 23 15:51:22.321422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 435/904] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:22.701176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:22.702192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:22.709534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.709981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.710972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.711431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.742998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32b5ae16-99f5-4d81-86dc-33ba92fadc0c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046741 May 23 15:51:22.743447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 470/905] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:22.853393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:22.854036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:22.860752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.861212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:22.861990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:22.862428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:23.028372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:23.038330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70f37f25-fb21-4b0e-932e-2415f4bc2e40 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.189231 May 23 15:51:23.038930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 436/906] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:23.335491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:23.335491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:23.335980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:23.336448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:23.337241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:23.337679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:23.489138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:23.497299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b50412c-f966-44b7-b0aa-9b85863d73b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1725 microversion: 2.1 time: 0.172986 May 23 15:51:23.497921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 471/907] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:23.754260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:23.755768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:23.761865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:23.762325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:23.763128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:23.763564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:23.792554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7e9f01e7-4324-4b90-a19d-0672b3253881 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042620 May 23 15:51:23.793110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 437/908] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:24.058630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:24.059361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:24.066236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.066676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.068053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.068494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.199436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:24.210276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ec56bf63-adb8-4177-b431-8c3e7d2043cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.153145 May 23 15:51:24.210276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 472/909] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:24.514843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:24.515417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:24.522040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.522494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.523296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.523847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.661292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:24.669577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10313c83-ab4a-4cd2-8cdc-bb9bc4bd221c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1702 microversion: 2.1 time: 0.158696 May 23 15:51:24.670521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 438/910] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1702 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:24.686173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1677e595-b3b3-479c-ac7a-c6a12b4147a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:24.693173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1677e595-b3b3-479c-ac7a-c6a12b4147a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.693720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1677e595-b3b3-479c-ac7a-c6a12b4147a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.717328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1677e595-b3b3-479c-ac7a-c6a12b4147a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Going to try to start instance {{(pid=124966) start /opt/stack/nova/nova/compute/api.py:2730}} May 23 15:51:24.805171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:24.806168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:24.814178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.814698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.815652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.816158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.846395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-713f89f2-7386-485f-8774-7e2820c8cdbe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046137 May 23 15:51:24.846945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 439/911] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:24.928349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-384764651"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:24.928884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:24.932780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1677e595-b3b3-479c-ac7a-c6a12b4147a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action" status: 202 len: 0 microversion: 2.1 time: 0.250480 May 23 15:51:24.933675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 473/912] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:51:24 2022] POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action => generated 0 bytes in 252 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:51:24.936115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.941631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.941631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.941631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.942118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:24.944154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:24.950697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.951171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:24.951943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:24.952377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.116850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.network.security_group_api [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Adding security group a09ae117-64f5-47c7-9644-49467f624b22 to port 7d2df762-e149-458d-aedd-407ed8492eb6 May 23 15:51:25.180502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:25.189715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b5b4cd6-518c-4501-a274-12320a0e9c47 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1711 microversion: 2.1 time: 0.251801 May 23 15:51:25.190465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 474/913] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1711 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:25.226552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:25.227497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:25.233903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.234710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.236161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.237186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.453818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:25.463232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-61790261-63e3-4503-8b73-6dcf30f4d42e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.241492 May 23 15:51:25.473165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 475/914] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:25.688227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc13ab8b-73bf-4fa7-85ca-565eb58f3fcc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action" status: 202 len: 0 microversion: 2.1 time: 0.764128 May 23 15:51:25.696864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 440/915] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:51:24 2022] POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action => generated 0 bytes in 773 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 15:51:25.697950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-8946f478-f0f3-46f7-9422-508e5cb9256a req-f3d8e499-88fa-4f64-b65c-4dbbf2a38e7b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:25.701421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:25.702055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:25.706014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-8946f478-f0f3-46f7-9422-508e5cb9256a req-f3d8e499-88fa-4f64-b65c-4dbbf2a38e7b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.706469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-8946f478-f0f3-46f7-9422-508e5cb9256a req-f3d8e499-88fa-4f64-b65c-4dbbf2a38e7b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.713695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.714632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.715711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.716418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.719739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-8946f478-f0f3-46f7-9422-508e5cb9256a req-f3d8e499-88fa-4f64-b65c-4dbbf2a38e7b service nova] Creating event network-changed:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 15:51:25.725381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-8946f478-f0f3-46f7-9422-508e5cb9256a req-f3d8e499-88fa-4f64-b65c-4dbbf2a38e7b service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031196 May 23 15:51:25.726189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 476/916] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:25.860130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:25.861468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:25.868699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.869392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.870170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:25.870612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:25.885939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:25.900253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73aba673-5311-420c-a139-4dda468c8162 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1760 microversion: 2.1 time: 0.202309 May 23 15:51:25.901153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 441/917] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1760 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:25.912930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9431c1a2-141c-426a-8fbb-4731ea0180cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058117 May 23 15:51:25.914256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 477/918] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:26.206835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:26.207402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:26.214339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.214790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.215585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.216024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.352049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:26.360005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3956ab5c-5054-4d0a-bd25-551d5f42f63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1711 microversion: 2.1 time: 0.157070 May 23 15:51:26.360573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 442/919] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1711 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:26.481630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:26.482321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:26.496299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.497116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.499255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.499810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.659517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:26.668040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af39e192-f497-492b-864d-1a66da1b464b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.190560 May 23 15:51:26.668604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 478/920] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:26.925139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:26.926140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:26.932879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.933361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.934182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:26.934618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:26.963174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2bad916e-4a87-47a2-b064-74dd89393de9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041962 May 23 15:51:26.963725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 443/921] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:27.377731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:27.378344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:27.384360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.384805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:27.385561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.385989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:27.519832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:27.530560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1cb390df-cb85-4dfe-a27b-1dbc55115454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1711 microversion: 2.1 time: 0.157022 May 23 15:51:27.531197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 479/922] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1711 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:27.686483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:27.687134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:27.694552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.695010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:27.696551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.696551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:27.857410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:27.866389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c85db71-c96d-40fd-b760-7ef16cbd9e9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.183733 May 23 15:51:27.866977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 444/923] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:27.974973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:27.975918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:27.982016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.982463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:27.983272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:27.983771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:28.014948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33150952-de20-423b-bae0-713ec4db13b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044059 May 23 15:51:28.015540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 480/924] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:28.547575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:28.548191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:28.554717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:28.555191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:28.555948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:28.556380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:28.687421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:28.696230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4962f0a-0135-4f37-9db1-8016cb146d91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1711 microversion: 2.1 time: 0.152412 May 23 15:51:28.696797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 445/925] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1711 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:28.883827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:28.884376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:28.890763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:28.891258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:28.892027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:28.892466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:29.027298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:29.028482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:29.031771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:29.040566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b6b41b8f-f7e8-4e5c-a883-1d6e396a0103 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.160589 May 23 15:51:29.041235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 481/926] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:29.042582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:29.043047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:29.043863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:29.044301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:29.076558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68ac1d74-d7a8-4c2a-a8b8-54c42291aa4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053480 May 23 15:51:29.077115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 446/927] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:29.713357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:29.714010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:29.720257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:29.720713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:29.721483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:29.721929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:29.848933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:29.857054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbdac606-705d-42a0-9904-740471628625 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1715 microversion: 2.1 time: 0.147633 May 23 15:51:29.857614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 482/928] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1715 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:30.057459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:30.058070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:30.064565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:30.065021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:30.065774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:30.066206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:30.084492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:30.088845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:30.089889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:30.097212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:30.097212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:30.097645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:30.098088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:30.134275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a28b16a1-78fe-46ab-9e3c-eb771817b401 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049360 May 23 15:51:30.134860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 483/929] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:30.201706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:30.210244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a6d09c6-4aac-411f-84e9-f0c74b76dd31 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.156532 May 23 15:51:30.210850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 447/930] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:31.146572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:31.147689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:31.154300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:31.154752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:31.155576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:31.156018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:31.185701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ea1bc492-78b9-4071-b669-7b39ee7977e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043751 May 23 15:51:31.186248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 484/931] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:31.227577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:31.228201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:31.235465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:31.236295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:31.236947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:31.237392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:31.366423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:31.374630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-88a36d6b-fa74-4c5d-bf52-0e6048ba2e1e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.150925 May 23 15:51:31.375209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 448/932] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:32.197786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:32.198709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:32.205140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:32.205585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:32.206335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:32.206766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:32.225003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:32.237767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7cfa110d-c07d-4c51-8847-d773739e35e6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043890 May 23 15:51:32.239655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 485/933] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:32.392978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:32.393509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:32.396518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:32.401668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:32.402116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:32.402869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:32.403333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:32.536794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:32.544777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6160d675-9a20-490a-a40c-678a9948a9fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.156091 May 23 15:51:32.545340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 449/934] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:33.249937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:33.251142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:33.258870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:33.259567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:33.260127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:33.260564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:33.289944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-834487ea-770b-4cb4-b14e-9c43542437c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043955 May 23 15:51:33.290503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 486/935] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:33.564954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:33.565578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:33.572740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:33.573197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:33.573964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:33.574404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:33.704808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:33.713348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4d9057ce-e3ff-4749-98b8-ba12bd26f6e5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.152456 May 23 15:51:33.713924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 450/936] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:34.302870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:34.303594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:34.310784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:34.311928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:34.312046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:34.312597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:34.342764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9e72f8f-8b2a-4c37-86d3-5edda5c88e56 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044888 May 23 15:51:34.343373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 487/937] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:34.730794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:34.731413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:34.738088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:34.738541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:34.739848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:34.740284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:34.873682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:34.882381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0e4cf73-35c5-4822-a5e0-765d510175bb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.155501 May 23 15:51:34.882970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 451/938] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:35.354195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:35.355171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:35.361933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:35.362389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:35.363263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:35.363725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:35.394414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c3e8498f-c8ec-4677-bc99-539fdad59825 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044289 May 23 15:51:35.394980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 488/939] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:35.899592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:35.900282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:35.908147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:35.908602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:35.909381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:35.909823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:36.050918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:36.059232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0f1f6446-edd0-4e61-99a6-66d04b91ded6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.163493 May 23 15:51:36.059811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 452/940] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:36.406294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:36.407276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:36.414620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:36.415082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:36.415888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:36.416334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:36.445972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c87df804-208d-4a6f-9745-95a2ffbef867 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043605 May 23 15:51:36.446463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 489/941] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:37.076610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:37.077226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:37.083689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:37.084136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:37.084894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:37.085324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:37.215725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:37.224627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5502974-57fb-44a1-bfd1-62d38fc79d02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151876 May 23 15:51:37.227016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 453/942] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:37.459748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:37.460609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:37.462881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:37.467562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:37.468013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:37.468768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:37.469196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:37.499016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-018669c2-1683-4af4-8883-6d37dbecadc4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043801 May 23 15:51:37.499598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 490/943] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:38.246659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:38.246659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:38.253514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.253956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.254724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.255177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.386565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:38.396357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1aa7d29-28fe-42d5-bb9c-449523bc87ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1717 microversion: 2.1 time: 0.154453 May 23 15:51:38.396932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 454/944] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1717 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:38.452472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:38.452971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:38.460011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.460609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.461241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.461663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.511062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:38.512098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:38.518226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.518682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.519475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.519914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.551185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e918fa25-832d-4310-82b1-d85b11a392d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044080 May 23 15:51:38.551783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 455/945] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:38.587698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Cannot 'detach_volume' instance 3e090a96-ef41-4548-ac75-e4dd2f371539 while it is in vm_state rescued May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:51:38.589313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:51:38.590980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Cannot 'detach_volume' instance 3e090a96-ef41-4548-ac75-e4dd2f371539 while it is in vm_state rescued May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3e090a96-ef41-4548-ac75-e4dd2f371539 in vm_state rescued. Cannot detach_volume while the instance is in this state. May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:51:38.592363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 3e090a96-ef41-4548-ac75-e4dd2f371539 while it is in vm_state rescued May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 3e090a96-ef41-4548-ac75-e4dd2f371539 while it is in vm_state rescued May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:51:38.593671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 409 to user: Cannot 'detach_volume' instance 3e090a96-ef41-4548-ac75-e4dd2f371539 while it is in vm_state rescued {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:51:38.594686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e253bab0-33cd-4526-9c52-6fe8ac01a522 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 409 len: 152 microversion: 2.1 time: 0.142261 May 23 15:51:38.594686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 491/946] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:51:38 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 152 bytes in 143 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 23 15:51:38.601039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:38.601887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:38.608574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.609029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.609801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.610239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.829689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03e5395d-722a-4af7-9f08-40d67a3109af tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action" status: 202 len: 0 microversion: 2.1 time: 0.233010 May 23 15:51:38.830518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 456/947] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:51:38 2022] POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:51:38.839420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:38.840041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:38.846989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.847513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:38.848276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:38.848714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:39.028085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:39.036312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-941d1360-0ac4-45bb-bdd8-66dd55e61e22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1725 microversion: 2.1 time: 0.200969 May 23 15:51:39.036881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 492/948] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1725 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:39.562904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:39.563959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:39.571257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:39.572042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:39.573403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:39.574158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:39.606288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1ed37f5-2e04-4711-8071-9f9c9a29d4c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048116 May 23 15:51:39.607010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 457/949] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:40.055380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:40.056206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:40.064930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:40.065469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:40.066670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:40.067346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:40.205000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:40.214097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-42e3d030-3d7a-45f4-9a43-1ec89f445edc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1725 microversion: 2.1 time: 0.164239 May 23 15:51:40.214667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 493/950] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:51:40.618062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:40.618946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:40.625366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:40.625816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:40.626868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:40.627333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:40.665094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae8b1c53-f4b7-4b49-8ede-40b9c8f3182b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051072 May 23 15:51:40.665679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 458/951] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:41.232199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:41.232989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:41.240551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:41.240996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:41.241792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:41.242201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:41.381708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:41.390265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-23c87b75-29d9-43b4-b8dd-23dd6430cd3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1725 microversion: 2.1 time: 0.162139 May 23 15:51:41.390885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 494/952] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1725 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:41.676665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:41.677680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:41.684315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:41.684768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:41.685537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:41.685976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:41.716692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d4c1ddcd-bbaa-4567-91b0-a5266d939ed0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044034 May 23 15:51:41.717261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 459/953] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:42.409805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:42.410669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:42.420204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:42.420801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:42.421835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:42.422421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:42.555501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:42.563541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6c1f3f9e-38a0-4973-8f1e-3abad3bf81c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1725 microversion: 2.1 time: 0.159247 May 23 15:51:42.564104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 495/954] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1725 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:42.728781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:42.729837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:42.736426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:42.736883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:42.737653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:42.738091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:42.767697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2c21520c-89f7-4ca0-be23-e0371c938625 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042772 May 23 15:51:42.768177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 460/955] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:43.581045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:43.581743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:43.589062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:43.589540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:43.591065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:43.591065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:43.734032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:43.742448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-103dc52b-8029-4ecc-bee5-8aed31f63d2c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1725 microversion: 2.1 time: 0.165211 May 23 15:51:43.742992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 496/956] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:43.779695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:43.780533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:43.787848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:43.788332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:43.789121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:43.789569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:43.824000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3cd681cf-0f1d-41aa-9d1e-c147cd686b1c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048434 May 23 15:51:43.824569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 461/957] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:44.760439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:44.761105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:44.767791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.768244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.769011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.769448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.836464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:44.837435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:44.849916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.850412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.855013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.855013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.882741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8f7ed0e8-0e29-4f13-a622-4ee619d450f2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050881 May 23 15:51:44.883354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 462/958] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:44.910566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:51:44.919373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28655192-4893-4837-91ab-d680f9a8d1e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1731 microversion: 2.1 time: 0.162767 May 23 15:51:44.919944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 497/959] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:51:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1731 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:51:44.936718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:44.937216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:44.943509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.943956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.944726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:44.945161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:44.949195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:45.025240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-812fed3a-5813-453d-a4ef-71501a684fdc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.092451 May 23 15:51:45.026421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 463/960] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 15:51:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-interface => generated 285 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:45.895410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:45.897233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:45.905379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:45.905830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:45.906633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:45.907062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:45.937466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4d19d19c-9af1-4341-b041-c6014bc4e515 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046389 May 23 15:51:45.938253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 498/961] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:46.010675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-d89b0826-2e25-41c2-847f-f722f7f44ab7 req-bc0821e5-aec4-4528-b364-04421cb20e47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:46.020619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d89b0826-2e25-41c2-847f-f722f7f44ab7 req-bc0821e5-aec4-4528-b364-04421cb20e47 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:46.021073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d89b0826-2e25-41c2-847f-f722f7f44ab7 req-bc0821e5-aec4-4528-b364-04421cb20e47 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:46.037262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-d89b0826-2e25-41c2-847f-f722f7f44ab7 req-bc0821e5-aec4-4528-b364-04421cb20e47 service nova] Creating event network-changed:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 15:51:46.041080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-d89b0826-2e25-41c2-847f-f722f7f44ab7 req-bc0821e5-aec4-4528-b364-04421cb20e47 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035306 May 23 15:51:46.042221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 464/962] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:51:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:46.949143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:46.950138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:46.956666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:46.957119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:46.957900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:46.958320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:46.991043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-12753054-ec4f-4b32-95e1-fec1abc349a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045861 May 23 15:51:46.991643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 499/963] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:48.003797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:48.004825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:48.011681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:48.012137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:48.012916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:48.013357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:48.044160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c04d795d-87cd-4cb0-a3da-25efed3989eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045052 May 23 15:51:48.044721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 465/964] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:48.055859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-bc6b5d4f-cee8-42ae-8e9c-e89271f3ba8a req-fa64d773-2e65-4f02-b5f3-13fccec0cb9b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:51:48.064563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-bc6b5d4f-cee8-42ae-8e9c-e89271f3ba8a req-fa64d773-2e65-4f02-b5f3-13fccec0cb9b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:48.065011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-bc6b5d4f-cee8-42ae-8e9c-e89271f3ba8a req-fa64d773-2e65-4f02-b5f3-13fccec0cb9b service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:48.076276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-bc6b5d4f-cee8-42ae-8e9c-e89271f3ba8a req-fa64d773-2e65-4f02-b5f3-13fccec0cb9b service nova] Creating event network-changed:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 15:51:48.082136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-bc6b5d4f-cee8-42ae-8e9c-e89271f3ba8a req-fa64d773-2e65-4f02-b5f3-13fccec0cb9b service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030492 May 23 15:51:48.083290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 500/965] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:51:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:49.055498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:49.056427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:49.063344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:49.063835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:49.064569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:49.065001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:49.095483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f86a7e38-b0ee-4a18-8e7b-82500ec33de3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044090 May 23 15:51:49.096064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 466/966] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:50.108906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:50.110224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:50.117983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:50.120886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:50.120886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:50.120886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:50.152732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-420b521f-b8a8-450a-8c55-dc548c86fbb6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048819 May 23 15:51:50.153296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 501/967] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:51.163779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:51.164717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:51.171716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:51.172216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:51.172993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:51.173430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:51.203961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35cdb29a-360b-4705-8b93-27579aa5b2fa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044119 May 23 15:51:51.204518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 467/968] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:52.216336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:52.217258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:52.223908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:52.224352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:52.225111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:52.225540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:52.251949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:51:52.256963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5397920e-6e10-4f1d-abe6-af17bb68bc68 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044757 May 23 15:51:52.257534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 502/969] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:53.269847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:53.270780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:53.278289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:53.278745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:53.279555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:53.280002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:53.313361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99b5a08f-e9a9-481a-8a7c-e2ce8504c3ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047440 May 23 15:51:53.313942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 468/970] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:54.326119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:54.327088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:54.333689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:54.334142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:54.334901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:54.335371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:54.365916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96c3c87a-d731-40a5-9494-d2a2d13135e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044059 May 23 15:51:54.366493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 503/971] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:55.378094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:55.379033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:55.385345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:55.385798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:55.386620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:55.387017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:55.421717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8a196be7-f294-4a44-9dff-268ab807a489 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047951 May 23 15:51:55.422292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 469/972] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:56.433647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:56.434578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:56.441697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:56.442734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:56.443532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:56.443967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:56.480090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6217c31-b780-466b-8822-cc0da940b580 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050302 May 23 15:51:56.480680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 504/973] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:51:57.492214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:57.493093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:57.499767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:57.500215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:57.500996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:57.501688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:57.531053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a11e6420-8298-4d48-99a1-d484bb5747bc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043608 May 23 15:51:57.531649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 470/974] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:58.542469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:58.543371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:58.549812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:58.550255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:58.551008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:58.551458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:58.579662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a7504b91-2eb0-4dfd-bc65-0ceebf6bfc2f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041157 May 23 15:51:58.580217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 505/975] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:51:59.591843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:51:59.592833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:51:59.599695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:59.600147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:59.600902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:51:59.601333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:51:59.631034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c8f13a8c-9057-460a-af7e-4332860345d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043067 May 23 15:51:59.631829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 471/976] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:51:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:00.642051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:00.642966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:00.649513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:00.649961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:00.650718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:00.651172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:00.680862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7f494c1c-f399-4f90-badc-db081748ce88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042794 May 23 15:52:00.681430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 506/977] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:01.692756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:01.693686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:01.700308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:01.700753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:01.701520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:01.701946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:01.731989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7d86caea-8ff2-4d11-9124-bc453c10ed10 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043324 May 23 15:52:01.732544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 472/978] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:02.743089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:02.744034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:02.750493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:02.750938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:02.751982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:02.752414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:02.783116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bcd22d7d-faab-4880-b09f-a8539cf06abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043874 May 23 15:52:02.783702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 507/979] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:03.794065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:03.795027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:03.797571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=48 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:03.803035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:03.803436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:03.804206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:03.804632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:03.834810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a70ee03-2364-4a65-af59-8fe8959cfba8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044687 May 23 15:52:03.835430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 473/980] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:04.846585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:04.848759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:04.855048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:04.855529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:04.856340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:04.856778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:04.889235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb3af939-1b2b-4845-beff-67ab8082b336 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046558 May 23 15:52:04.889796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 508/981] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:05.902024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:05.902863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:05.909920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:05.910375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:05.911169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:05.911612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:05.940209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a13f8993-79d7-400c-89cf-2aea27d5cd52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042484 May 23 15:52:05.941086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 474/982] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:05 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:06.952820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:06.953666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:06.959951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:06.960400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:06.961166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:06.961605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:06.992247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-448e196a-8201-4e58-afdc-2dc37fa9780a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043502 May 23 15:52:06.992809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 509/983] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:08.004741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:08.005674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:08.012165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:08.014018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:08.014018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:08.014018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:08.043753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e66bc4da-d9ce-4370-a760-524c5a7f93c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043748 May 23 15:52:08.044301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 475/984] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:09.055929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:09.056824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:09.059268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=40 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:09.063909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:09.064356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:09.065102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:09.065534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:09.096178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-df55b0ba-2e7e-4a7c-854f-85561fb94fbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044697 May 23 15:52:09.096735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 510/985] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:10.107888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:10.108817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:10.116020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:10.116466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:10.117218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:10.117650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:10.146544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b23120e-76a3-494b-b38d-c0cb48a7486f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042608 May 23 15:52:10.147083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 476/986] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:11.157857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:11.158786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:11.165897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:11.166343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:11.167118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:11.167553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:11.197113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c53389ef-aefb-4490-8983-dc4d704a3272 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043181 May 23 15:52:11.197660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 511/987] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:12.209285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:12.210215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:12.216657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:12.217107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:12.217873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:12.218314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:12.248303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-673b9a88-cae4-455e-8056-5ab03dfb3af7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043417 May 23 15:52:12.248859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 477/988] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:13.260195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:13.261154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:13.268200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:13.268656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:13.269520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:13.269961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:13.299766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad1554a5-c05c-412e-aede-90fd79dbf133 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043518 May 23 15:52:13.300335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 512/989] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:14.312033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:14.312956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:14.321111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:14.321563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:14.322536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:14.323222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:14.353422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9f3602ff-8fba-4c02-92dc-34318b5ae3b9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045305 May 23 15:52:14.353903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 478/990] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:15.365659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:15.366625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:15.373428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:15.373880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:15.374649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:15.375109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:15.404057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c39155d9-bc1a-494c-89e3-a9d4ae6d45c6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042936 May 23 15:52:15.404647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 513/991] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:16.416381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:16.417315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:16.423878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:16.424332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:16.425108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:16.425547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:16.428838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=89 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:16.457077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe52bc6e-709e-4e3d-9d28-9f41328e9e3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044542 May 23 15:52:16.457840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 479/992] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:17.469573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:17.470501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:17.477620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:17.478072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:17.478836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:17.479307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:17.509598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-822434f6-493d-49ac-8106-86607f9c36bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044031 May 23 15:52:17.510157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 514/993] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:18.521632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:18.522466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:18.529495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:18.529946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:18.530707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:18.531160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:18.559505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e1a64a42-aa3d-4e34-9bc4-d801c6bae9c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042436 May 23 15:52:18.560048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 480/994] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:19.571030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:19.571992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:19.578372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:19.578822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:19.579617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:19.580052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:19.608720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7d8b4583-ce58-4f55-8327-19caa31246b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041709 May 23 15:52:19.609265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 515/995] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:20.620446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:20.621371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:20.628430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:20.628905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:20.629671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:20.630680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:20.659635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1f6532a5-20e7-4f7c-b1eb-7dc6b6d5e79d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042972 May 23 15:52:20.660204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 481/996] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:21.670753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:21.671768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:21.678789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:21.683875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:21.683875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:21.683875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:21.709601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b48361c8-c772-4743-bfb4-c5234073c5ce tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042803 May 23 15:52:21.712568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 516/997] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:22.720905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:22.721824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:22.728773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:22.729218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:22.729969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:22.730402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:22.759779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa90e954-e42b-45fb-a392-765db1a2a860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042805 May 23 15:52:22.760339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 482/998] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:23.772011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:23.772931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:23.779426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.779876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:23.780718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.781149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:23.809695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ab8c36d-1ea8-4319-99bd-6be917370912 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041684 May 23 15:52:23.810244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 517/999] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:23.887202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:23.887706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:23.894080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.894528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:23.895310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.895745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:23.923954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d3982cf0-f6b5-42de-8f0e-144d484c13ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments/356edfc4-f82d-4a59-bd83-146727fd42c3" status: 200 len: 194 microversion: 2.20 time: 0.040805 May 23 15:52:23.924523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 483/1000] 10.222.0.32 () {60 vars in 1453 bytes} [Mon May 23 15:52:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments/356edfc4-f82d-4a59-bd83-146727fd42c3 => generated 194 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:23.991653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:23.995250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:23.998206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.998650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:23.999435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:23.999864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:24.079265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lazy-loading 'services' on Instance uuid 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:52:24.276153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:24.404645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af661cf2-72cd-4c19-ab98-12c8bd5902ae tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments/356edfc4-f82d-4a59-bd83-146727fd42c3" status: 202 len: 0 microversion: 2.20 time: 0.417228 May 23 15:52:24.405335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 518/1001] 10.222.0.32 () {60 vars in 1456 bytes} [Mon May 23 15:52:23 2022] DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments/356edfc4-f82d-4a59-bd83-146727fd42c3 => generated 0 bytes in 419 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 15:52:24.414497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:24.426478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:24.426478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:24.426478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:24.426478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:24.426889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:24.452965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5bd3c4e7-8d20-4051-a528-21eeb6760351 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042950 May 23 15:52:24.453748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 484/1002] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:24.822925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:24.823972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:24.831178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:24.831640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:24.832473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:24.832945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:24.865810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-268b9c16-d40f-456f-8c7c-3a89b9b7a606 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048188 May 23 15:52:24.866376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 519/1003] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:25.465191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:25.466070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:25.473261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:25.474262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:25.475038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:25.475522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:25.505531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1f6ec142-28d9-4230-a144-59c093cb964f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044397 May 23 15:52:25.506092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 485/1004] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:25.876420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:25.877599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:25.884023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:25.884467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:25.885215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:25.885644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:25.914668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9bd68c63-f39d-47d9-9026-39cd1bbfdb07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042158 May 23 15:52:25.915243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 520/1005] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:26.517610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:26.518459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:26.525598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:26.526048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:26.526812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:26.527288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:26.555805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4e111611-5741-4ed6-85aa-9b25bb33095d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042229 May 23 15:52:26.556354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 486/1006] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:26.926297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:26.927677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:26.934064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:26.934511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:26.935299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:26.935735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:26.964788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c0ff20b-e358-423d-8fd9-82931cb9e7c0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042523 May 23 15:52:26.965336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 521/1007] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:27.566873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:27.568451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:27.574925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:27.575402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:27.576167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:27.576599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:27.605608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53f6c7b3-34e1-4221-b675-c1d123b1af78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042639 May 23 15:52:27.606185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 487/1008] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:27.976831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:27.977698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:27.984181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:27.984633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:27.990388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:27.990388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:28.014863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2de01e6-047c-40b4-bd15-5164a310f6d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042521 May 23 15:52:28.015448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 522/1009] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:28.617313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:28.618151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:28.624646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:28.630121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:28.630121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:28.630121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:28.656447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd229150-0c54-41ce-a601-87cab3f0a6cc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043521 May 23 15:52:28.657033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 488/1010] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:29.026297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:29.027245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:29.033639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:29.036807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:29.036807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:29.036807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:29.064049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8f1d1783-2b1a-47da-8d8c-614f2d06d98e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041652 May 23 15:52:29.064592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 523/1011] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:29.668345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:29.669278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:29.675824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:29.676284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:29.677577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:29.678015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:29.710548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-969df5d5-03cb-4bed-b2fd-e43f780adfd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046149 May 23 15:52:29.711179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 489/1012] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:30.076323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:30.077387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:30.083843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:30.084296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:30.085110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:30.085552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:30.115725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a80e0b4-64cb-4fa7-8639-357bc028755e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043288 May 23 15:52:30.116285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 524/1013] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:30.722266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:30.723253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:30.730417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:30.730870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:30.731706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:30.732140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:30.761522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-38bd9ce6-cc0e-463c-924f-9e69322a7e42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043311 May 23 15:52:30.762069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 490/1014] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:31.127648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:31.128577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:31.135618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:31.136076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:31.136843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:31.137284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:31.165467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0b3ad553-f8de-4385-a63c-1f6c54050518 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042063 May 23 15:52:31.166019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 525/1015] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:31.774345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:31.775299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:31.781735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:31.782189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:31.782944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:31.783416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:31.813734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8cdb1a9f-14a5-47f1-b9f3-94dd4615298f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044126 May 23 15:52:31.814289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 491/1016] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:32.177177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:32.178022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:32.184568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:32.185020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:32.185781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:32.186289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:32.215737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02c7dc01-6b70-4630-8c94-1678f45a1ac6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042664 May 23 15:52:32.216293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 526/1017] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:32.825559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:32.826592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:32.834047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:32.834501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:32.835313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:32.835750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:32.865434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a20ad5a2-6969-4260-a756-23d8e9c79679 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043796 May 23 15:52:32.865989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 492/1018] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:33.227121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:33.228012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:33.234773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:33.235656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:33.236030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:33.236467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:33.265797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-741a98fa-9a9e-40c2-8828-d405b0c0bfad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042709 May 23 15:52:33.266347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 527/1019] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:33.877266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:33.878194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:33.884665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:33.885119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:33.885879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:33.886319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:33.914781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa30d9cd-50c5-4204-95c0-5ed2a12f1013 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041681 May 23 15:52:33.915361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 493/1020] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:34.277076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:34.278011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:34.284633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:34.285083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:34.285852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:34.286916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:34.315213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fb0a73a9-007c-4e75-b5cf-811c090a88a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042042 May 23 15:52:34.315770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 528/1021] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:34.926307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:34.927808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:34.934429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:34.934820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:34.935658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:34.936095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:34.964867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ded28167-ed33-450f-b7a5-e6236ff721af tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042505 May 23 15:52:34.965409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 494/1022] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:35.325950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:35.326871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:35.333319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:35.333766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:35.334523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:35.334955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:35.363387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aed01273-16d1-489d-b616-350da6cc3df2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041354 May 23 15:52:35.363976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 529/1023] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:35.978543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:35.979580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:35.981842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=44 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:35.986695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:35.992408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:35.992408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:35.992408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:36.013691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:36.018253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-79e0f861-5148-4cc9-8359-851d4e2dad02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043720 May 23 15:52:36.018808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 495/1024] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:36.375346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:36.376334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:36.382615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:36.383067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:36.383863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:36.384302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:36.415571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-780580b2-2f5a-470a-b6d8-bf50a44c1d4b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044288 May 23 15:52:36.416129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 530/1025] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:37.031329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:37.032264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:37.039903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:37.040360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:37.041133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:37.041575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:37.070703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cca5890b-bdb7-4b6a-8430-e333d309755c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043859 May 23 15:52:37.071291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 496/1026] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:37.428333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:37.429257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:37.435674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:37.436117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:37.437020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:37.437577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:37.467585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-94cfc217-d22f-489d-a452-5c2fe59d613a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043967 May 23 15:52:37.468139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 531/1027] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:38.081927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:38.083462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:38.090016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:38.090466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:38.091264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:38.091704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:38.120392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08b14146-1234-4e43-8de4-d9826ef5a22b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042447 May 23 15:52:38.120948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 497/1028] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:38.479415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:38.480332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:38.486829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:38.487243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:38.487997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:38.488426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:38.516981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a036ab45-503d-462c-a61a-c35544d9e1cf tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041669 May 23 15:52:38.517525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 532/1029] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:39.132391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:39.133381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:39.139867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:39.140310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:39.141107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:39.141550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:39.170112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4b9defa6-f140-48ec-b692-8a4d703606f6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041670 May 23 15:52:39.170668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 498/1030] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:39.528557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:39.529423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:39.531911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=46 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:39.536733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:39.537188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:39.537955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:39.538394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:39.568735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a72d0dd1-a9bd-4df8-902f-80b21f81ea67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044234 May 23 15:52:39.569295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 533/1031] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:40.182023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:40.182937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:40.189402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:40.189849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:40.190606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:40.191041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:40.221543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8954207d-b6ce-45a1-93b6-f56f486fa569 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043507 May 23 15:52:40.222106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 499/1032] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:40.580455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:40.581376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:40.588598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:40.589054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:40.589851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:40.590288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:40.620147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-44ba3424-5895-44e8-b798-04ab126201de tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043594 May 23 15:52:40.620704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 534/1033] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:41.234461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:41.235557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:41.247570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.248024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.248792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.249255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.266707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:41.283307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2d1d9c2f-2424-430f-a09d-85d115a8ffc7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053177 May 23 15:52:41.284168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 535/1034] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:41.322262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:52:41.322907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:41.328854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.329304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.330293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.330728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.358915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lazy-loading 'flavor' on Instance uuid 909fad86-0068-4016-bf26-7246f0b13e53 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:52:41.635380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:41.635380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:41.639964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.640420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.641238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.641698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.674929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b15e9150-9952-4249-b30a-011eeda0bf14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046779 May 23 15:52:41.675545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 536/1035] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:52:41.685161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-923c2c6b-63ab-4fa9-b08f-01fe9cfa7c17 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "POST /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53/action" status: 202 len: 0 microversion: 2.1 time: 0.479754 May 23 15:52:41.686089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 500/1036] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:52:41 2022] POST /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53/action => generated 0 bytes in 482 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:52:41.700321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:41.700938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:41.707158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.708129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.708908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:41.709348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:41.842544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:41.852283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-acaf8465-7290-40f7-bf50-9ae389a05c5a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.161897 May 23 15:52:41.852856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 537/1037] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:42.298437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:42.299517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:42.305747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.306196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:42.306953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.307423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:42.338365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89a45fc7-edaf-4811-8e8e-7b234e5d1e4b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044699 May 23 15:52:42.339041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 501/1038] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:42.686049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:42.687050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:42.694343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.694794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:42.695597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.696030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:42.724592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7ed61a2e-300a-4883-96e6-9a2798e55c17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042578 May 23 15:52:42.725135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 538/1039] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:42.870399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:42.870927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:42.877333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.877814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:42.878573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:42.879010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:43.241297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:43.248959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c26279c9-89a9-49b9-947e-5c0d161ffbb2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.382503 May 23 15:52:43.249558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 502/1040] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:43.351651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:43.352567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:43.358780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:43.359262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:43.360017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:43.360447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:43.389787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c4f2b833-b412-47ae-b0c1-77c727fb1455 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042725 May 23 15:52:43.390342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 539/1041] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:43.735753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:43.736693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:43.743351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:43.743810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:43.744582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:43.745023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:43.775857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55be1ee0-82ff-4657-800b-02a791559197 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043989 May 23 15:52:43.776335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 503/1042] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:44.265900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:44.266518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:44.273004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.273465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.274233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.274671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.400688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:44.401548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:44.407959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.408419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.409198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.409651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.413178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:44.420702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d3bce3f8-a96f-46cb-a7c9-b37e1550fcb4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.158758 May 23 15:52:44.421267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 540/1043] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:44.440104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ee186923-74f0-4ef5-923e-20cfc6edd2d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043487 May 23 15:52:44.440659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 504/1044] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:44.786599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:44.787554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:44.793565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.794019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.794782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:44.795246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:44.824413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e7dc85e-e76b-4a66-8ed8-2d50db6b866a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041777 May 23 15:52:44.824961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 541/1045] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:45.438926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:45.439502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:45.446322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.446499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.447258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.447699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.451797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:45.452798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:45.459518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.460044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.460910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.461419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.491858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-81f182b0-3a2a-4534-8211-8e13da757c00 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044117 May 23 15:52:45.492437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 542/1046] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:45.578424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:45.586545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cff8f809-9b7f-4f02-891d-0698d779adb5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151818 May 23 15:52:45.587130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 505/1047] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:45.836599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:45.837520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:45.843754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.844202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.844961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:45.845392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:45.873402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-becdd913-d042-41bc-b0b6-c83e8e7c0c7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041306 May 23 15:52:45.873957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 543/1048] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:46.504008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:46.504939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:46.511459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.511907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.512698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.513141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.541888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4598f7d2-cd49-4ce5-9554-239c24754e2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042151 May 23 15:52:46.542436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 506/1049] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:46.604068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:46.604685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:46.611271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.611728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.612491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.612937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.741024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:46.748569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c64af0b0-cef2-4a88-a91f-331d7009612a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.148325 May 23 15:52:46.749103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 544/1050] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:46.884809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:46.885665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:46.891892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.892350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.893125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:46.893561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:46.923334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b3328aa-c6b4-4513-b0ea-e73a56f2925d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042442 May 23 15:52:46.923890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 507/1051] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:47.553662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:47.554600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:47.560693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.561143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.561992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.562430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.592062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f5846f89-45cb-48b8-adbd-5b542d949286 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042513 May 23 15:52:47.592618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 545/1052] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:47.765845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:47.766440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:47.773009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.773461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.774230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.774669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.902280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:47.909960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-370e3909-a063-4900-9ba7-78429177f1c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.148035 May 23 15:52:47.910541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 508/1053] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:47.933798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:47.934772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:47.941273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.941717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.943062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:47.943512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:47.972890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b94d34d3-a8fe-450c-a073-0a6787be6eda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043005 May 23 15:52:47.973947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 546/1054] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:48.604481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:48.605449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:48.611601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.612049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.612815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.613253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.642976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5df93a78-c80c-455a-8317-47782fe7a078 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042477 May 23 15:52:48.643563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 509/1055] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:48.761945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:52:48.762902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:48.769285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.769732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.771175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.771615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.926269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:48.926937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:48.933885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.934339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.935178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:48.935624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:48.951368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:48.986477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b1e66b35-1a15-488b-9b40-571f2c3fc64e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action" status: 202 len: 0 microversion: 2.1 time: 0.228148 May 23 15:52:48.991666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 547/1056] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:52:48 2022] POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:52:48.998071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:48.999236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:49.006419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.010852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.010852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.010852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.041387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a9337f8-b9d7-45d4-befb-cc9c013c5c41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048336 May 23 15:52:49.042164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 548/1057] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:49.047057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:49.047848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:49.053724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.054167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.054931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.055394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.078168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:49.086215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd917ad8-f2cc-4a04-8761-a499ff123c8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.163875 May 23 15:52:49.086800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 510/1058] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:49.245237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:49.255706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32ef6183-02eb-4f31-9d77-3f1129904654 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1752 microversion: 2.1 time: 0.212545 May 23 15:52:49.256701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 549/1059] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1752 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:49.655497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:49.656360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:49.663718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.664181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.664965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:49.665408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:49.697378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6cb75e32-eba9-4bc7-b302-9a56716184fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046184 May 23 15:52:49.697962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 511/1060] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:50.052939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:50.053903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:50.060759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.060923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.061651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.062084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.091740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eeaa8b89-8790-42b1-a4ca-77498498fd07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042737 May 23 15:52:50.092295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 550/1061] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:52:50.105748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:50.106377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:50.113321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.113829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.114603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.115047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.255583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:50.263075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-30ef63c3-1d98-4bac-9fc8-317f986324c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.162406 May 23 15:52:50.263662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 512/1062] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:50 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:50.278476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:50.279157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:50.285173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.285619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.286468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.286905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.290974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:50.428161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:50.436116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4429649b-9d7b-4379-95ac-2a5965349592 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.161458 May 23 15:52:50.436683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 551/1063] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:50.709302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:50.710153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:50.716212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.716665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.717429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:50.717865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:50.746386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73457bdd-7309-4a40-b110-b72c9b1f4821 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041165 May 23 15:52:50.746934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 513/1064] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:51.103351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:51.104291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:51.110709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.111197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.112001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.112447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.143283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1340815c-c0d7-4c27-9c95-b3aac09e904d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043972 May 23 15:52:51.143854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 552/1065] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:51.281016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:51.281634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:51.288667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.289117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.289880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.290316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.421519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:51.429681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-11b01fb1-0ecc-4c61-9213-fc6b77f93e94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.152579 May 23 15:52:51.430251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 514/1066] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:51.453563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:51.454185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:51.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.460732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.461498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.461934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.606163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:51.613121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36d1ff69-2b11-4ec9-ab6b-a6f3271e7b16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.163416 May 23 15:52:51.614188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 553/1067] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:51.761149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:51.762088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:51.768188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.768748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.769548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:51.769992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:51.799273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99e587e1-6af7-4613-b3b9-20f1ed3e5d5c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044872 May 23 15:52:51.799827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 515/1068] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:52.154063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:52.155654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:52.162167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.162620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.163418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.163855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.194218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ea78998a-7b4c-4071-ac96-6b978c5cd532 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044115 May 23 15:52:52.194793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 554/1069] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:52.447456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:52.448068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:52.454864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.455351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.456129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.456568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.630650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:52.631320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:52.637207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.637650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.638463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.639081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.671309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:52.679456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ec3af498-17c2-4586-867e-e3b6f3898f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.235913 May 23 15:52:52.680028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 516/1070] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:52.804269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:52.812314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9587bbe-3ee0-4e6f-a153-201fe646d07c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.185756 May 23 15:52:52.812518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:52.812882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 555/1071] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:52.813481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:52.819759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.820214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.820987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:52.821424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:52.852651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b81817ef-ecdd-47ca-a03e-09493d94f8a1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044181 May 23 15:52:52.853204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 517/1072] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:53.205912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:53.206756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:53.213779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.214232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.214992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.215454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.244912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b22250ae-e69a-487a-9314-12e3215983a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043004 May 23 15:52:53.245475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 556/1073] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:53.697514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:53.698131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:53.704611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.705067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.705831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.706270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.828942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:53.829628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:53.835875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.836323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.837086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.837513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.847533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:53.855033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4de855d2-cf69-4083-86fb-7d8ba3a8f386 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.161408 May 23 15:52:53.855757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 518/1074] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:53.865562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:53.866502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:53.872781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.873248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.874020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:53.874459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:53.904287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e427a7d1-473d-4c30-8b9d-bf88ceffa04a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044050 May 23 15:52:53.905141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 519/1075] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:53.983703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:53.992482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47717011-04ff-405a-a6e0-25cc47d167aa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.167472 May 23 15:52:53.993174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 557/1076] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:54.257202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:54.258170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:54.264202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.264657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.265516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.265961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.294750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-479761e2-a7a3-4a51-a400-1f628e96b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041575 May 23 15:52:54.295345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 520/1077] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:54.871997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:54.872613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:54.879870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.880325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.881092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.881529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.916328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:54.917245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:54.924280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.924737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.925520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:54.925958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:54.956133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d0c9e5c-cfbd-483e-a428-081d5d704243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043715 May 23 15:52:54.957243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 521/1078] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:55.011586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:55.012209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:55.015710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:55.018576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.019087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:55.020511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.020960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:55.025245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5d600454-0d88-4126-9769-b6c31ecc8cf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.157072 May 23 15:52:55.025831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 558/1079] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:55.162671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:55.170499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-12b5e27a-c080-4607-b956-339601ee43e0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.163162 May 23 15:52:55.171062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 522/1080] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:55.306401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:55.307385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:55.313807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.314259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:55.315028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.315518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:55.344588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9e0bf2e0-ef4d-49c4-a598-4f8812a27997 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042142 May 23 15:52:55.345149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 559/1081] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:55.968999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:55.969904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:55.976071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.976530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:55.977299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:55.977734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.008439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-736f6ac0-76d9-406f-b76b-704e9c19e4fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043560 May 23 15:52:56.009031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 523/1082] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:56.042294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:56.042897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:56.049290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.049734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.050485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.050913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.188032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:56.188780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:56.195858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.196316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.197092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.197535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.200409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:56.208554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3b304e76-528d-43de-912b-491f0d397dbb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.170057 May 23 15:52:56.215243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 560/1083] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:56.338461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:56.346594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-05156fba-fcef-41bf-973f-77a323f721c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.162413 May 23 15:52:56.349175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 524/1084] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:56.356184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:56.357131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:56.363072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.363595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.364902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:56.365346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:56.394451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f0c3c2c4-b991-4d70-a011-776e8ff097aa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042179 May 23 15:52:56.395021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 561/1085] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:52:57.020959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:57.021892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:57.028022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.028480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.029334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.029776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.058631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0aa3d75-141f-4b98-9b4c-8d594b0d41d2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042210 May 23 15:52:57.059211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 525/1086] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:57.225970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:57.226614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:57.233191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.233643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.234537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.234984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.364879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:57.365505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:57.367656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:57.372162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.372618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.373482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.373917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.375260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db40ff65-775f-4d13-bd4f-98bb65a4294f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.153089 May 23 15:52:57.375840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 562/1087] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:57.405759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:57.406688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:57.412841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.413221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.413989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:57.414415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:57.444515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4b25d13-ae45-4200-a497-36434758d38c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042532 May 23 15:52:57.445314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 563/1088] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:52:57.518180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:57.526172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a509e1b9-fd49-464a-9e26-1b51b0863f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.165842 May 23 15:52:57.526738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 526/1089] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:58.070744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:58.071717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:58.077831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.078284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.079057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.079521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.119427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e899c72-8d38-4f60-8d1f-8d147d653a84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053304 May 23 15:52:58.120005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 564/1090] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:52:58.393158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:58.393807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:58.401574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.402061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.402830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.403298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.405924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:58.456234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:58.457173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:58.463784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.464242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.465014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.465464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.496674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-df615a34-24f4-4c20-91ff-499fdee62ee3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044390 May 23 15:52:58.497265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 565/1091] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:52:58.543235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:58.543858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:58.550017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.550460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.552157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:58.552276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:58.658359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:58.666389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d0f339a-3f90-4626-bb3c-fe82a5d512d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.277328 May 23 15:52:58.667049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 527/1092] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:58.785991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:58.794039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d8dbeb29-3a38-4f13-b5c3-0a21a19e21bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.254597 May 23 15:52:58.794633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 566/1093] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:52:59.132966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:59.133881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:59.140201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.140657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.141588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.141974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.175220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b33dd5d3-4c8f-4f28-85a5-deb01eacc66e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046291 May 23 15:52:59.175791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 528/1094] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:52:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:52:59.510328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:59.511609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:59.519628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.521098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.522574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.523344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.557074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:59.561801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-087e4186-8572-47c9-a129-0cbc06da1c8d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055972 May 23 15:52:59.562370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 567/1095] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:52:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:59.595776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-643d4ec7-2a02-4b39-b0f4-209629455f7f req-ecb8335a-5962-4cee-b870-c502e65efacf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "name": "network-vif-unplugged", "status": "completed", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:52:59.604535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-643d4ec7-2a02-4b39-b0f4-209629455f7f req-ecb8335a-5962-4cee-b870-c502e65efacf service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.604978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-643d4ec7-2a02-4b39-b0f4-209629455f7f req-ecb8335a-5962-4cee-b870-c502e65efacf service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.616740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-643d4ec7-2a02-4b39-b0f4-209629455f7f req-ecb8335a-5962-4cee-b870-c502e65efacf service nova] Creating event network-vif-unplugged:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 15:52:59.622130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-643d4ec7-2a02-4b39-b0f4-209629455f7f req-ecb8335a-5962-4cee-b870-c502e65efacf service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029969 May 23 15:52:59.622683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 529/1096] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:52:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:52:59.684049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:59.684659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:59.691264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.692291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.693068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.693664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.827548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:52:59.836334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aeb88122-5481-4222-86c5-f2232a9dfeb0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.156533 May 23 15:52:59.837117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 568/1097] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:52:59.842110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:59.842892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:59.849667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.875705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.875705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.875705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.914735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:52:59.915469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:52:59.922770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.923249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:52:59.924014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:52:59.924446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.003823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:00.011768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9c7f81ee-85ae-4d64-993a-b690d648f277 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.173368 May 23 15:53:00.012352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 569/1098] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:52:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:53:00.187012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:00.187966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:00.194050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.194501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.195307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.195744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.224974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-85381de6-4df0-4248-9cf8-feb412cebdbb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041986 May 23 15:53:00.225735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 570/1099] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:53:00.575382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:00.576336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:00.584663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.585112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.585870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.586300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.615440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7762bac0-43df-42a4-8701-797d7f8ff42b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044007 May 23 15:53:00.620288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 571/1100] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:53:00.853770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:00.854306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:00.860667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.861117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.861881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:00.862321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:00.991034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:00.998798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b6d457a-3963-44ae-9057-621735cb4c6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.148992 May 23 15:53:00.999417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 572/1101] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:01.029235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:01.029890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:01.036856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.042752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.042752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.042752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.181599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:01.189849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5b8fb1ff-3d06-4200-861d-1c7dbd62eeab tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.164455 May 23 15:53:01.190415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 573/1102] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:01.236688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:01.237661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:01.243678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.244138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.244899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.245334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.275061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a167917-0843-42cf-b40a-5ee8724699c1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042328 May 23 15:53:01.275642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 574/1103] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:53:01.630310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:01.631321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:01.638046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.638491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.639284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:01.639711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:01.668675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a521de05-e539-4ca7-88ef-7b3557566366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045580 May 23 15:53:01.672121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 575/1104] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:53:02.016408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:02.017021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:02.023089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.023559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.024314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.024842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.152115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:02.159514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6bb4f8a7-d10a-4023-9d28-eb55e84f1bf5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.146993 May 23 15:53:02.160079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 576/1105] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:02.206552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:02.207238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:02.213241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.213689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.214447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.214877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.355932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:02.364590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ea7f565-31f4-4f27-98b1-9c6c1f418406 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.161738 May 23 15:53:02.365159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 577/1106] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:02 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:02.369697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:02.370603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:02.376383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.376838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.377856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.378289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.406375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-69b46d01-5847-44fe-b86c-c390ed1fc010 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040205 May 23 15:53:02.406928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 578/1107] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:02.680497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:02.681428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:02.687978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.688433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.689195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:02.689633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:02.718420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5722f5cf-2f38-43ba-b1a7-c23a2912d207 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041954 May 23 15:53:02.718976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 579/1108] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:53:03.176520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:03.177191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:03.183351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.183794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.184541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.184971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.313575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:03.321538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7de68b8b-9a81-4fc9-bfde-184d596d2f63 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.148926 May 23 15:53:03.322099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 580/1109] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:03.382335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:03.382971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:03.388899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.389347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.390105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.390540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.529239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:03.537196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a950f84-d134-4a33-ad22-816b0d4419df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.159151 May 23 15:53:03.538169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 581/1110] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:03.542527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:03.543743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:03.549830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.550281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.551600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.552030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.582539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5a85219e-d5ea-45fd-af84-8f57c76cc865 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043581 May 23 15:53:03.583148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 582/1111] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:03.730191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:03.731056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:03.733430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:03.738156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.739285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.740038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:03.740467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:03.769462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b85a1efc-0f82-4ebd-a1fe-97e441c9c51b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043299 May 23 15:53:03.770052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 583/1112] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:53:04.339324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:04.339936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:04.346144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.352526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.352526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.352526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.366237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:04.482913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:04.490558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9bf2822-6b2b-47c3-8d64-415f0a2b060b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.155377 May 23 15:53:04.491046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 584/1113] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:04.553648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:04.554255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:04.560240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.560687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.561439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.561862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.701767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:04.710448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0ada567e-5ba6-4af2-b5f9-8f178cd8dca0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.160655 May 23 15:53:04.711017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 585/1114] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:04.715540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:04.716443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:04.722150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.722586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.723367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.723797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.757019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04577a2e-8c8b-4d65-8e5f-f00aade76806 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044871 May 23 15:53:04.757644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 586/1115] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:04.780904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:04.781813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:04.787629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.788071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.788818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:04.789253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:04.817921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe717bb3-a918-40fa-91e0-1a43f487e5d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040755 May 23 15:53:04.818467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 587/1116] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 15:53:05.508062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:05.508590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:05.514600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.515047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.515843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.516279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.650184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:05.657791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ea9dce70-766f-4962-ae2e-8806434723f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.153641 May 23 15:53:05.658361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 588/1117] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:05.728303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:05.728931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:05.735297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.735747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.736515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.736947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.878641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:05.886848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fadff6e4-c830-41ca-b1dc-613596092c30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1757 microversion: 2.1 time: 0.162376 May 23 15:53:05.887461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 589/1118] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:05.892058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:05.893008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:05.899366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.899827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.900591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.901028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.935058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0f0ea09c-1613-41f5-b9b2-4f8f7445239c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046539 May 23 15:53:05.935940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 590/1119] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:05.940525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:05.941464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:05.947183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.947639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.948404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:05.948841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:05.978389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ceecc11-8f1c-43e6-87bb-b0b0177571f8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041411 May 23 15:53:05.978957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 591/1120] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:05 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:06.679601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:06.680131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:06.686218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:06.686663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:06.687454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:06.687890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:06.816696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:06.824579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7438c51b-f03f-4275-a7b2-c1ab2be075fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.149378 May 23 15:53:06.825156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 592/1121] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:06.904590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:06.905322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:06.911881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:06.912323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:06.913075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:06.913504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.058985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:07.067153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5b19f46-3bae-4623-9871-f8d9845b0558 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1760 microversion: 2.1 time: 0.166378 May 23 15:53:07.067760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 593/1122] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:06 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1760 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:07.072457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:07.073387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:07.079600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.080090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.080838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.081280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.111624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ed20aa7-df76-4050-b1d3-6ed31cf02812 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042774 May 23 15:53:07.112194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 594/1123] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:07.116896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:07.117810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:07.123455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.123907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.124667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.125105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.153993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5d1621b-4a42-4b15-98a7-83fba0caa5d9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040619 May 23 15:53:07.154636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 595/1124] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:07.842320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:07.842814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:07.849531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.849983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.850802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:07.851381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:07.979964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:07.987849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fb8b8487-9144-421c-8997-f7749b156deb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.149882 May 23 15:53:07.994390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 596/1125] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:08.123933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:08.124773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:08.131936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:08.132382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:08.133135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:08.133564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:08.163278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0bbb883a-95c3-4102-9b5d-41a213f1613b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043314 May 23 15:53:08.163852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 597/1126] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:53:08.168472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:08.169418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:08.176487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:08.176935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:08.177695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:08.178130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:08.207310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76010a4f-05af-47b6-8d8f-8ae1b464a38d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042411 May 23 15:53:08.207871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 598/1127] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:09.008985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:09.009665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:09.016075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.016522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.017302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.017715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.148600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:09.156376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-169bfaf8-3f29-4686-89fc-acc10d3ed56c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151227 May 23 15:53:09.156952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 599/1128] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:09.174609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:09.175583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:09.182104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.182562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.183357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.183796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.212481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3d625bca-27fa-4059-96ee-06b0f62bfe7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041703 May 23 15:53:09.213142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 600/1129] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 15:53:09.217840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:09.218773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:09.225386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.225863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.226621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:09.227079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:09.230801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:09.258026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-75b1154f-1213-4a4f-9f55-a17372c7e9d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043796 May 23 15:53:09.258587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 601/1130] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:09.918835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=71 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:09.944315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=17 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:10.174575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:10.175242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:10.182612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.183054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.183838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.184275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.313846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:10.321954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-456e967c-63c1-4eca-a37a-a4ed30f19c2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151372 May 23 15:53:10.322515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 602/1131] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:10.327386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:10.328223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:10.334096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.334542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.335332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.335767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.364969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f8b64b9-03f8-48f5-9b44-abedb0b03699 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041444 May 23 15:53:10.365519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 603/1132] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:10.369949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:10.370849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:10.376514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.376965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.377991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:10.378428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:10.407726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-daae79d3-4f20-44eb-a295-554973febbdd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041184 May 23 15:53:10.408290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 604/1133] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:11.339425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:11.340161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:11.347210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.347665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.348475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.348921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.482274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:11.489962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-602c00ae-e905-4aa9-b503-5d84cb37aadc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.154420 May 23 15:53:11.490475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 605/1134] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:11.495246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:11.496168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:11.502095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.502812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.503604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.504576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.533829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8ca579f8-2d39-4200-a6a4-58f2b4468d19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042330 May 23 15:53:11.534384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 606/1135] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:11.539026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:11.539958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:11.545538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.545986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.546787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:11.547210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:11.575794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aa24244c-a906-4349-9edc-e5c64c4b702e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040414 May 23 15:53:11.576350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 607/1136] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:12.507838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:12.508578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:12.515409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.515855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.516608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.517038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.647482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:12.661693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f600cb52-5a9f-4673-8c58-20b9ec676828 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.154590 May 23 15:53:12.661693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 608/1137] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:12.664578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:12.665497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:12.672221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.672643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.673402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.673829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.704477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0c55af71-2546-41d1-8dec-01c1eb7250e4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044205 May 23 15:53:12.705028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 609/1138] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:12.710310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:12.711249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:12.717389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.717834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.718580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:12.719008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:12.747224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-062c0b56-c3ed-45dc-a265-274febe0a4e1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041184 May 23 15:53:12.747779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 610/1139] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:13.676689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:13.677339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:13.683827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.684268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.685018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.685446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.814513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:13.822033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6edfa484-04dd-4519-9185-5be2e31028c6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.149771 May 23 15:53:13.822594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 611/1140] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:13.827304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:13.828242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:13.834376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.834823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.835611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.836044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.865297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce50aea8-c2ce-4b2a-a315-cfb4787004d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041590 May 23 15:53:13.865772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 612/1141] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:13.870212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:13.871140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:13.876738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.877183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.878200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:13.878630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:13.907149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0d2a8c5-750c-4f49-99db-d9589697d6dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040350 May 23 15:53:13.907711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 613/1142] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:14.838706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:14.839381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:14.845851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:14.846289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:14.847039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:14.848176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:14.979045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:14.986640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9b107e0c-c5ae-4a60-a83a-fa7dd0e1f017 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151776 May 23 15:53:14.987224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 614/1143] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:14.991770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:14.992677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:14.998481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:15.009630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:15.009630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:15.009630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:15.036349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1aac7bb8-ab33-4475-a004-a4674c35e59e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048061 May 23 15:53:15.036928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 615/1144] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:15.041777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:15.043998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:15.048947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:15.049391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:15.050150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:15.050578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:15.080622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f64d367e-7ef5-42d1-9e53-5a3e589ec027 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042653 May 23 15:53:15.081181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 616/1145] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:16.004025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:16.004650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:16.011201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.011655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.012421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.012858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.143993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:16.147916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:16.152256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a07e11dc-8fc3-4aa2-8a0b-1873dcf0a756 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.152447 May 23 15:53:16.152825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 617/1146] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:16.157411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:16.161739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:16.165031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.165474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.166225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.166654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.195906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0c52b88-c216-4304-bb28-0a965ebfb677 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042072 May 23 15:53:16.196462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 618/1147] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:16.202051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:16.202051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:16.210820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.211529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.212824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:16.213691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:16.247370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a0eb09b-2605-4331-922b-a20ac15a6c7c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049731 May 23 15:53:16.247960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 619/1148] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:17.169938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:17.170622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:17.177522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.177964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.178720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.179173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.314566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:17.322242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-928279b8-c9aa-4c10-ad9d-507c45d751ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.156172 May 23 15:53:17.322808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 620/1149] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:17.327403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:17.328320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:17.334160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.334609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.335400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.335838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.366834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-737bb70e-77f6-4964-a85b-e882ca53003d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042938 May 23 15:53:17.367431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 621/1150] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:17 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:17.372016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:17.372929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:17.378929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.379411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.380168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:17.380608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:17.409159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c64bcaf1-8fcb-4ab1-9c70-0861e621e177 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040697 May 23 15:53:17.409719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 622/1151] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:18.339059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:18.339744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:18.346450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.346892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.348304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.348735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.476566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:18.484113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f0f63a0-eae4-4b4a-9ded-7be742d52a00 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.148869 May 23 15:53:18.484675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 623/1152] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:18.489175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:18.490085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:18.495737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.496187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.496944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.497390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.526560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5de8b65-5776-41bb-8d98-74ef6d0249fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040854 May 23 15:53:18.527138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 624/1153] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:18.531592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:18.532543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:18.538827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.539304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.540059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:18.540492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:18.568443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c072c1d-c393-4624-ab72-9809b2e086ed tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040296 May 23 15:53:18.568999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 625/1154] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:19.500938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:19.501586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:19.508299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.508750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.509512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.509958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.639308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:19.646428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-27d30d4e-c2b7-40dc-b9c3-10df522fafe2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.149443 May 23 15:53:19.647196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 626/1155] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:19.651854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:19.652772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:19.658543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.659046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.659844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.660783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.690549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3337791c-5a10-4d15-8c97-f69fc09fc49f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042292 May 23 15:53:19.691153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 627/1156] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:19.695787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:19.696707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:19.702695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.703210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.706971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:19.706971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:19.733241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-27c478cb-daef-4c53-b37a-324e4d5a36db tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041023 May 23 15:53:19.737590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 628/1157] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:20.663707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:20.664410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:20.670850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.671444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.672214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.672649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.805067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:20.812799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3af67598-fe13-4838-97b6-a48b5e36d77f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.152875 May 23 15:53:20.813370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 629/1158] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:20.818677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:20.819689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:20.825439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.825894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.826659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.827140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.855330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7163c478-2277-4d04-be18-59915766a85e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040829 May 23 15:53:20.855814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 630/1159] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:20.860427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:20.861328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:20.867533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.867982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.868749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:20.869183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:20.897943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bee21fd3-5c03-4453-b8f5-97b32335977f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041080 May 23 15:53:20.898497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 631/1160] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:21.830677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:21.831375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:21.840141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:21.840585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:21.841353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:21.841787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:21.972550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:21.980051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-88bcfdc4-3b55-4ef0-bfa1-c112a8119680 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.153850 May 23 15:53:21.980619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 632/1161] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:21.985284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:21.986197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:21.987901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:21.992643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:21.993090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:21.995678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:21.995678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:22.025478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7fbd5591-2eca-49c5-8d75-4f6218d5e459 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043825 May 23 15:53:22.026041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 633/1162] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:22.030681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:22.031616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:22.037467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:22.037917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:22.038681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:22.039141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:22.055258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:22.068014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-129f63ae-76c7-4e1b-8486-5a67c79ae3a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040941 May 23 15:53:22.068564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 634/1163] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:22.997627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:22.998284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:23.005156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.005608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.006374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.006806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.139711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:23.147691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe5b7351-0095-400b-8ec9-3a50bd634c7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.154238 May 23 15:53:23.148264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 635/1164] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:23.153017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:23.153934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:23.159957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.160984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.161756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.162224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.190512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28cc584d-7a3d-4cb7-b84c-3a3d419dfcbd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041234 May 23 15:53:23.191074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 636/1165] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:23.195559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:23.196467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:23.202221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.202672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.203467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:23.203909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:23.232988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-12f55485-9809-4361-9703-2e02e3843cba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040851 May 23 15:53:23.233734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 637/1166] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:24.164761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:24.165441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:24.171918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.172365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.173120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.173556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.306336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:24.313865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8860a277-cbd0-4a80-9d54-e8f024a4208a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.153002 May 23 15:53:24.314419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 638/1167] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:24.319494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:24.320404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:24.326077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.326514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.327298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.327732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.356883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0960a38b-3966-42bc-8a4a-d50f2e54c943 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041462 May 23 15:53:24.364875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 639/1168] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:24.364875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:24.364875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:24.368851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.369293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.370042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:24.370467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:24.399709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7c38f428-b482-49cc-af34-4d4b9a7ce7d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041276 May 23 15:53:24.400262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 640/1169] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:24.584019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:53:24.584584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 15:53:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 15:53:25.330749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:25.331352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:25.338227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.338673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.339465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.339905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.500119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:25.508679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b183f2b8-53f6-4fd3-b9f6-3b57277acd7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.181788 May 23 15:53:25.509254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 641/1170] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:25.514028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:25.514985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:25.521837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.522290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.523059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.523519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.559211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd87ec9f-1cc0-4d77-8a0b-05961cc5692e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047829 May 23 15:53:25.559798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 642/1171] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:25.564537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:25.565462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:25.572185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.572641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.573411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:25.573849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:25.610117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b4e79200-1598-4630-be5d-1f11c0ba6798 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049143 May 23 15:53:25.610688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 643/1172] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:26.529454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:26.530076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:26.536540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.536980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.537748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.538183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.668646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:26.676741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-074ab8a6-c6c1-4d95-bfa2-88bc7f6a5103 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151205 May 23 15:53:26.677312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 644/1173] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:26.681876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:26.682787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:26.688489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.688935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.689696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.690131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.731852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5bf5ea2d-4726-48c3-aa22-53547e98432e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053473 May 23 15:53:26.732517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 645/1174] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:26.737222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:26.738687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:26.746121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.746890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.748249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:26.748993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:26.780572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d647f4bf-3a52-4299-ab95-d4c5fdfdd542 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046902 May 23 15:53:26.781037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 646/1175] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:27.695033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:27.695736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:27.702204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.702655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.703551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.703990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.837064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:27.844889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d2fcea76-402d-44ea-bc8a-aba8ba892d84 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.154385 May 23 15:53:27.845462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 647/1176] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:27.850690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:27.851618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:27.857380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.857826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.858592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.859032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.888915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-578d90cd-230b-4c14-98ca-c7cf529c3c38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042427 May 23 15:53:27.889478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 648/1177] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:27.894131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:27.895034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:27.900673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.901131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.901885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:27.902323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:27.935685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-969b8867-8cc6-46e1-9e3c-97d8cf43d444 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044904 May 23 15:53:27.936197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 649/1178] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:28.862408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:28.863696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:28.870172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:28.870622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:28.871418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:28.871856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:29.005202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:29.012833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cfeefa6c-8a30-434f-8280-c817fc10ed3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.154293 May 23 15:53:29.023089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 650/1179] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 15:53:29.023089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:29.023089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:29.026089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:29.026543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:29.027343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:29.027785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:29.031601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:29.058668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1edc3303-6536-49cf-bd8c-a4a1d0bfa320 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043734 May 23 15:53:29.059239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 651/1180] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:29.063869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:29.064771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:29.071025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:29.071501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:29.072251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:29.072685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:29.101287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9905fbc-5859-4f07-bc4b-ffca3ab09e4a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041016 May 23 15:53:29.101849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 652/1181] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:53:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:53:30.029894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:30.030519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:30.037761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.038220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:30.038994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.039463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:30.056013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:53:30.176905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:53:30.184664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-780bc6f0-02fa-493b-9b6c-2bebb9aba7cb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.158706 May 23 15:53:30.192127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 653/1182] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:53:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:53:30.196892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:30.197808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:30.204444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.204894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:30.205652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.206428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:30.236112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dcd79d37-d61d-4279-bbaf-75ba80fdb978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042729 May 23 15:53:30.236670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 654/1183] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:53:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:53:30.242146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:53:30.242655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:53:30.248781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.249229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:30.249989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:53:30.250421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:53:40.069159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:40.246483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=45 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:53:40.268862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=26 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:55.192630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:54:55.236692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-28797cab-1db9-46c2-8dad-0ca73286b313 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 202 len: 0 microversion: 2.1 time: 115.436450 May 23 15:54:55.239736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 530/1184] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:52:59 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 0 bytes in 115441 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 15:54:55.247963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:54:55.250655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:54:55.256249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.256712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.258443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.258943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.289440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5703720c-acc7-45f9-a505-9d2b9f9106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048039 May 23 15:54:55.290002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 531/1185] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:54:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:54:55.294417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:54:55.295029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:54:55.301170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.301625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.302387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.302825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.439609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:54:55.447158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-655eb08f-7754-4e7a-b5e0-f2ed97fba7cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.156179 May 23 15:54:55.447724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 532/1186] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:54:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:54:55.452310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:54:55.462700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:54:55.462700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.462700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.462700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.463107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.493077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d8ca1b2-3753-4a9a-ac50-db67b55c33de tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044300 May 23 15:54:55.493654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 533/1187] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:54:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:54:55.498167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:54:55.502573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:54:55.505150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.505596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:55.506344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:54:55.506775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:54:57.954754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:54:57.955460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 15:54:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 15:55:05.196857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:05.502577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=8 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:05.524882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:15.921462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:55:15.922056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 15:55:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 15:55:18.743937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:55:18.744585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 15:55:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 15:55:30.356585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de94e0df-f583-4b0b-8342-b237ee7a602f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 202 len: 0 microversion: 2.1 time: 120.118723 May 23 15:55:30.357544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 655/1188] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:53:30 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 0 bytes in 120120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:55:30.364220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:55:30.364821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-384764651"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:55:30.365701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:55:30.372547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:55:30.372927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:55:30.373698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:55:30.374128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:55:30.515770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.network.security_group_api [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Removing security group a09ae117-64f5-47c7-9644-49467f624b22 from port 7d2df762-e149-458d-aedd-407ed8492eb6 May 23 15:55:31.024636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33867f11-c5e2-439f-b63a-79c08ca25d40 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action" status: 202 len: 0 microversion: 2.1 time: 0.665446 May 23 15:55:31.025257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 656/1189] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 15:55:30 2022] POST /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/action => generated 0 bytes in 667 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:55:31.030894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:55:31.031440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:55:31.044809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:55:31.044809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:55:31.045572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:55:31.046013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:55:40.368991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:41.036650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:55:41.072102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:43.916997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:56:43.917625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 15:56:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 15:56:44.592628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:56:44.593182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 15:56:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 15:56:55.340266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:56:55.343390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 15:56:55.344776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:56:55.345435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:56:55.347157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:56:55.348934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:56:55.352146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) May 23 15:56:55.353460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:56:55.354714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6c5c3ad8-d713-4d37-87ff-89594934b350) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:56:55.354714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9eb467bb-c549-41fd-83c5-1c7816bfbb29 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 119.850272 May 23 15:56:55.354714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 534/1190] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:54:55 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 119851 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:56:55.450528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:56:55.454847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:56:55.458387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.459389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.460157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.460596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.480159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:56:55.490447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ca784fbc-c518-45df-83b4-a79c3ce5ed3f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.143880 May 23 15:56:55.491013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 535/1191] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:56:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:56:55.495416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:56:55.496026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:56:55.501793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.502242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.503088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.503460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.635855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:56:55.643330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8fab4f8a-0b29-4bdd-ae75-cb62c279274c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151329 May 23 15:56:55.643891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 536/1192] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:56:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:56:55.720051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:56:55.720982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:56:55.727777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.728230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.729004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.729443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.758055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7de24bc1-34a0-40d3-93c5-f77641d7fc0e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.113097 May 23 15:56:55.758534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 537/1193] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:56:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:56:55.762961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:56:55.763481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:56:55.768896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.769347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:56:55.770104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:56:55.770544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:57:05.485389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:05.766793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=8 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:05.788965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=22 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:30.501749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 15:57:30.504115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 15:57:30.505516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:57:30.506673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:57:30.507834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:57:30.511025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:57:30.512360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) May 23 15:57:30.513726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 15:57:30.514984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1f7a9fd8-8607-4802-b4c7-543e45975f85) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:57:30.514984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3cf4a913-a960-46c8-90d1-a0f97c6f7afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 119.479260 May 23 15:57:30.514984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 657/1194] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:55:31 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 119480 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:57:30.514984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:57:30.515385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:57:30.518969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:57:30.519438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:57:30.520197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:57:30.520626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:57:30.552228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:57:40.518895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:40.540746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:40.556416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:17.964573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:58:17.965731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 15:58:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 15:58:35.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:58:35.930999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 15:58:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 (ip 10.222.0.32) !!! May 23 15:58:38.752070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 15:58:38.752712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 15:58:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 (ip 10.222.0.32) !!! May 23 15:58:55.591798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:58:55.677217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 15:58:55.679461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 15:58:55.680950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:58:55.681588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 15:58:55.683023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:58:55.684701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:58:55.685739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) May 23 15:58:55.686836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 15:58:55.689098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-21e17898-f620-4003-b223-915bf2490d12) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 15:58:55.689098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2916cf14-42d6-446b-93de-a4601369ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 119.921496 May 23 15:58:55.689098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 538/1195] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:56:55 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 119922 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 15:58:55.746890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-a1570030-0b73-48c1-8e85-8ebe88c746a5 req-18634747-e35c-40e9-bae1-46caa9793aca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 15:58:55.755863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-a1570030-0b73-48c1-8e85-8ebe88c746a5 req-18634747-e35c-40e9-bae1-46caa9793aca service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:58:55.756314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-a1570030-0b73-48c1-8e85-8ebe88c746a5 req-18634747-e35c-40e9-bae1-46caa9793aca service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:58:55.767928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-a1570030-0b73-48c1-8e85-8ebe88c746a5 req-18634747-e35c-40e9-bae1-46caa9793aca service nova] Creating event network-changed:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 15:58:55.773360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-a1570030-0b73-48c1-8e85-8ebe88c746a5 req-18634747-e35c-40e9-bae1-46caa9793aca service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090767 May 23 15:58:55.774210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 539/1196] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 15:58:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:58:55.779211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:58:55.779799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:58:55.786334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:58:55.786725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:58:55.787514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:58:55.787963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:05.596345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:05.783360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:05.805478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=6 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:30.814816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 15:59:30.834093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b101d41e-1d19-402e-b9c5-f8a169961b65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 202 len: 0 microversion: 2.1 time: 120.326456 May 23 15:59:30.840985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 658/1197] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:57:30 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 0 bytes in 120328 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 15:59:30.841258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:59:30.842351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:59:30.848666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:30.849124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:30.849891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:30.850329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:30.879137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24ce97bc-336a-40be-af23-6881f4f587eb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043010 May 23 15:59:30.879706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 659/1198] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 15:59:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 15:59:30.884929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:59:30.885538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:59:30.890906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:30.891395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:30.892153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:30.892592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:31.024382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 15:59:31.031988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8dc440cb-f8fa-4ce6-83bb-6a12c9af1f35 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.151352 May 23 15:59:31.032553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 660/1199] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 15:59:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 15:59:31.037066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:59:31.037999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:59:31.044307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:31.044760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:31.045522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:31.045963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:31.074315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e47ce1e9-30cd-48ed-82e1-674049cade45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040785 May 23 15:59:31.074853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 661/1200] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 15:59:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 15:59:31.079224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 15:59:31.079714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 15:59:31.085002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:31.085457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:31.086210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 15:59:31.086673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 15:59:40.818807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:41.083594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=8 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:41.104712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:03.924710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:00:03.925328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:00:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 16:00:04.600936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:00:04.601481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:00:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 16:00:55.925803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:00:55.928378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:00:55.930040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:00:55.930959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:00:55.932519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:00:55.934237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:00:55.935606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) May 23 16:00:55.937473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48294534-26ea-42ce-9b5e-fda7d664fc08) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 120.154726 May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:00:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:00:55 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (10.222.0.32) May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: CRITICAL nova [None req-88b380ed-7d36-4d23-abf7-7f75b8a0c510 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Unhandled error: OSError: write error May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova OSError: write error May 23 16:00:55.939536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova May 23 16:00:55.940336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 540/1201] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:58:55 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 0 bytes in 120157 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 23 16:00:56.038680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:00:56.081996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:00:56.083800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:00:56.089881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:00:56.090327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:00:56.091088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:00:56.091545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:06.043611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:06.087845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:06.109018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:30.912364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:01:30.914696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:01:30.919462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:30.920354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:30.921509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:30.922702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:30.923771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) May 23 16:01:30.924905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab756ab9-25ed-4939-af64-51d26d6ba478) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 119.841037 May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:30 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (10.222.0.32) May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: CRITICAL nova [None req-d9011f06-edf6-408b-83e7-cdba904462ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Unhandled error: OSError: write error May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova OSError: write error May 23 16:01:30.926103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova May 23 16:01:30.926771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 662/1202] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 15:59:31 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 0 bytes in 119843 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 23 16:01:30.926771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:30.926771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:30.930612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:30.931064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:30.931897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:30.932364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:30.968940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.012996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.043225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:01:31.044093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:01:31.045736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.046812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.048195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.049415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.050537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) May 23 16:01:31.051821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82790173-ac3e-4245-bef5-e6a36294e0ca) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 35.113157 May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:01:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 (ip 10.222.0.32) !!! May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:01:31 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 (10.222.0.32) May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: CRITICAL nova [None req-21f117c9-1378-4432-b72b-6d8e3cd11d80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Unhandled error: OSError: write error May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova OSError: write error May 23 16:01:31.053063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova May 23 16:01:31.053646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 541/1203] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:00:55 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 0 bytes in 35115 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 23 16:01:31.053646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.053646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.059058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.059538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.060306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.060724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.075345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:01:31.077811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:01:31.080400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.081681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.083751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.086020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.089538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) May 23 16:01:31.091715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0f00e2f-f090-4192-9960-45e5b0f6a5ab) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.160101 May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (ip 10.222.0.32) !!! May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:31 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a (10.222.0.32) May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: CRITICAL nova [None req-7c661619-b0f7-42b6-af2d-6d700cdaeb95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Unhandled error: OSError: write error May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova OSError: write error May 23 16:01:31.093844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova May 23 16:01:31.094687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 663/1204] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:30 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 0 bytes in 162 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) May 23 16:01:31.094687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.094687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.097908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.098359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.099168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.099296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051267 May 23 16:01:31.099711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.099882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:01:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments (ip 10.222.0.32) !!! May 23 16:01:31.099882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:01:31 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments (10.222.0.32) May 23 16:01:31.100258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: CRITICAL nova [None req-b20184f9-03d8-45be-ba79-af1a1452d7e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Unhandled error: OSError: write error May 23 16:01:31.100258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova OSError: write error May 23 16:01:31.100258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova May 23 16:01:31.100511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 542/1205] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 0 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) May 23 16:01:31.105611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.111720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.112300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.112742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.113516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.113930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.142421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d8b06cf0-3410-4200-aa58-2fd8dbc4483f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041026 May 23 16:01:31.142981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 543/1206] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:01:31.147522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.148004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.154154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.154595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.155386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.155820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.232130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:01:31.238178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.239904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-86c21658-ca7d-4064-a635-7766c0e57148 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.157661 May 23 16:01:31.240476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 664/1207] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:01:31.245590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.246081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.251798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.252240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.252991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.253418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.298834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:01:31.301066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:01:31.302602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.303603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.304803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.306164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.307970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) May 23 16:01:31.309592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.310876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-da7dfa85-1db9-4c81-b708-2e24156e3fd9) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:31.310876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4fa22cab-2a9c-426f-ac44-efcfba7055dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 0.158680 May 23 16:01:31.310876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 544/1208] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:31 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 160 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:01:31.310876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.311281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.316754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.317201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.317961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.318392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.346815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.401594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.438839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:01:31.441032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:01:31.442543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.459804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.461144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.462351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) May 23 16:01:31.464013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:01:31.465458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97024d3-f41b-474e-89e0-57afdc837eec) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:31.465458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-694dfd0c-4bfe-4e26-a66e-650a0d1e34b1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.201048 May 23 16:01:31.465458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 665/1209] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:31 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 202 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:01:31.465458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.465795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.465795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.465795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.465795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.466262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.480777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:01:31.484848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:01:31.486191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.486874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:01:31.488138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.489260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.491346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) May 23 16:01:31.493189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:01:31.495677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d744fb39-716b-408b-a385-f1d66aa399a9) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:01:31.495677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7476ba2b-69b8-4cac-8770-d414946400f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.182252 May 23 16:01:31.495677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 545/1210] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:31 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 183 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:01:31.495677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.497426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.499066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34d1689a-967f-4a03-b3ef-c3a9f82a86e0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054749 May 23 16:01:31.499918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 666/1211] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:01:31.501309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.501780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.502575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.504725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.505300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.505480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.512722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.513513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.514879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.515794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.607114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:01:31.681742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:01:31.688856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca1e234c-709e-47f4-ae97-ea4ff3d20bd2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.187605 May 23 16:01:31.689429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 667/1212] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:01:31.694003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.694976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.700671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.701126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.701888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.702318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.736044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ea202b40-679e-4a17-8054-c20c25dba4d9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045446 May 23 16:01:31.736703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 668/1213] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:01:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:01:31.741339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:31.745470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:31.749350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.749805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.750639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:31.751076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:31.826175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR oslo.messaging._drivers.impl_rabbit [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [25d4d26c-ffcb-418e-a94d-5772f6611074] AMQP server on 10.222.0.32:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 16:01:32.834827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [25d4d26c-ffcb-418e-a94d-5772f6611074] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 50100. May 23 16:01:32.837945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd7604f2-66ac-4da2-876e-d786372600f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 202 len: 0 microversion: 2.1 time: 1.349720 May 23 16:01:32.839857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 546/1214] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:31 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 0 bytes in 1351 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:01:32.843714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:01:32.844298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:01:32.850629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:32.851082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:32.851866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:01:32.852301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:01:37.973680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:01:37.974213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:01:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 (ip 10.222.0.32) !!! May 23 16:01:41.746789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=14 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:41.772997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=44 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:42.847638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=12 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:42.870921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=45,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:55.940231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:01:55.940726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 16:01:58.761065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:01:58.761709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:01:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 (ip 10.222.0.32) !!! May 23 16:02:56.106243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.147553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:56.189209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:56.190956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.192249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.194171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.196860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.198117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) May 23 16:02:56.201548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.202856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f200c8f4-62d0-46eb-9fdf-b49500c2d5c4) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:56.202856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d2da1491-52e8-452b-992f-8c544431cd53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 84.448297 May 23 16:02:56.202856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 669/1215] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:31 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 84449 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:56.322986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.324211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.332922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.333658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.334422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.334857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.365712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbb28e23-1054-4b5c-8d09-098d9e5b0681 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.178188 May 23 16:02:56.366293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 670/1216] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:56.450851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:56.453179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:56.454603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.455296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.456877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.458052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.459192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) May 23 16:02:56.460281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.461325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-79b76c81-9d3c-433d-bc65-2657f3188287) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:56.461325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7260ecbd-71b7-4045-851b-d7033fe045da tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 83.614968 May 23 16:02:56.461325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 547/1217] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:01:32 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 83616 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:56.461325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.461650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.467408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.467854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.468622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.469055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.485174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.485710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.492072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.492523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.493295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.493911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.551254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.590844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.617628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 HTTP/1.1 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:56.619879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:56.621355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3a14c169-ee02-4abd-8bae-be450c73183c', 'id': '58902dfa-8f68-4cae-a849-ef582736a197'} May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.622207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.623257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.625394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) May 23 16:02:56.626483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.627581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1c46a927-66ae-4f47-a64e-626f07434f38) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:56.627581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-158895e2-57da-4c98-8d8d-997954653234 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 400 len: 269 microversion: 2.1 time: 0.165240 May 23 16:02:56.627581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 548/1218] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:56 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 269 bytes in 166 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:56.627816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.628440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.634665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.635137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.636171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.636603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.651681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:56.653823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:56.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.656034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.658683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) May 23 16:02:56.663628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:56.664793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f463386-fc28-4c25-95a4-d1ced711ab15) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:56.664793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f91de7b8-063b-4e9c-9214-564c62a56649 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.287977 May 23 16:02:56.664793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 671/1219] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:56 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 289 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:56.772926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:02:56.775339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.776349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.780579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f675ac5d-3d86-4598-a194-69d63260c5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1561 microversion: 2.1 time: 0.157868 May 23 16:02:56.781147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 549/1220] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:02:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1561 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:02:56.783172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.783701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.784519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.785504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.785992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-2a2f7eeb-15d1-4854-bb9f-f4fc012dac4d req-a34f12d7-b323-4ff8-a797-baf91bca0844 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:02:56.793664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2a2f7eeb-15d1-4854-bb9f-f4fc012dac4d req-a34f12d7-b323-4ff8-a797-baf91bca0844 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.794102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2a2f7eeb-15d1-4854-bb9f-f4fc012dac4d req-a34f12d7-b323-4ff8-a797-baf91bca0844 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.805364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-2a2f7eeb-15d1-4854-bb9f-f4fc012dac4d req-a34f12d7-b323-4ff8-a797-baf91bca0844 service nova] Creating event network-changed:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 16:02:56.810662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-2a2f7eeb-15d1-4854-bb9f-f4fc012dac4d req-a34f12d7-b323-4ff8-a797-baf91bca0844 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028412 May 23 16:02:56.811283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 550/1221] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:02:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:56.814581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33f550ad-f2c7-4490-b548-40cb5c0e0cba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.157324 May 23 16:02:56.815291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 672/1222] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:02:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:56.818638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.819241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.821141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:56.821636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:56.825358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.825806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.826564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.826994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.827514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.827969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.828737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:56.829173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:56.912877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.955029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:56.985649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:56.988078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:56.989408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.990248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:56.991279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.992394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.993385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) May 23 16:02:56.999708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.001119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35f42364-7ca3-48c8-a178-e706340dcc5f) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.001119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5ade301-ac1b-4cb2-979b-ef4624c5c465 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 0.176029 May 23 16:02:57.001119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 551/1223] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:56 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 178 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.001119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.001433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.004789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.005247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.006028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.006469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.027951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:57.030200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:57.031644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.032262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.033816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.034978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.036109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) May 23 16:02:57.037216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.040591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16a86806-fc4f-4f10-ab99-fa236d7ed133) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.040591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0ffcffa2-b851-42b3-854c-cba981a8b916 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.215530 May 23 16:02:57.040591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 673/1224] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:56 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 217 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.041063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b4d2517d-ab03-45e3-a61c-2846fdec75c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046413 May 23 16:02:57.041063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 552/1225] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:57.041361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.041361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.042710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.043285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.047634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.050654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.050654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.050888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.050998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.051136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.051136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.051136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.133191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.174783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.207547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:57.209703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:57.211408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.212275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.213396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.214529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.215569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) May 23 16:02:57.216640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.217665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-396de3fd-1b69-40e2-8976-c3ec39ad016b) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.217665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a3c47f91-67e1-4ad5-adb9-af374f2abb20 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.177058 May 23 16:02:57.217665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 674/1226] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 181 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.223030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.223665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.229793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.230672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.231466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.231912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.263533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 HTTP/1.1 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:57.265669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:57.267322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3a14c169-ee02-4abd-8bae-be450c73183c', 'id': '58902dfa-8f68-4cae-a849-ef582736a197'} May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.268066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.269408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.270874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.279617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) May 23 16:02:57.281258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.282729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-392d815d-c417-4bd0-bd62-1be3c7288bb7) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.282729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af31fe16-8474-4e59-afbb-fa62da058909 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 400 len: 269 microversion: 2.1 time: 0.228028 May 23 16:02:57.282729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 553/1227] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 269 bytes in 229 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.282729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.283266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.283266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.283266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.283266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.283592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.311138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e3126d4e-48b0-4211-92ea-827f4c9f6be4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042466 May 23 16:02:57.311701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 554/1228] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:57.316963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.317441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.323835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.324295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.325048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.325481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.409807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.445387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:02:57.452960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1e232cbf-2451-4022-b85e-d2bab03a5bd9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1561 microversion: 2.1 time: 0.233759 May 23 16:02:57.453609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 675/1229] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1561 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:02:57.459421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.459913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.466230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.467391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.467848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:57.468750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:57.470286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.473245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.474264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) May 23 16:02:57.475394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.476430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.476511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fb83c13e-d752-49c8-a432-51874d54e9f8) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.476604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.476684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9fa0b535-07a7-492a-923c-01438ce7543b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 0.157557 May 23 16:02:57.476684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 555/1230] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 158 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.476684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.476684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.482616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.483070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.483875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.484315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.516083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1de1c6db-ca19-437f-81e3-650a3439094e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044338 May 23 16:02:57.516634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 556/1231] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:57.521114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.521593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.527028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.527509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.528261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.528693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.559138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.607709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.635423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:57.637813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:57.639277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.640153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.641274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.642385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.644674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) May 23 16:02:57.645972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.647666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3ff4c092-f366-4717-99d3-5e4469ad9dbd) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.647666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-48251d20-b29b-4b9a-b109-2fb9302c61ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.183782 May 23 16:02:57.647666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 676/1232] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 186 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.647666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.648052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.650783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.651263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.652012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.652439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.669673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:57.671933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:57.674370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.675868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.678064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.680207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.681944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) May 23 16:02:57.683199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.684554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c1dc2081-a7df-4cd1-84df-d301746f0284) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.684554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9dc3e827-9ea0-48b5-850e-d6bbdaa2cda1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.155800 May 23 16:02:57.684554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 557/1233] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 157 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.684554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.685055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.686175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.686623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.687415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.687847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.733449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.788739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 HTTP/1.1 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:57.791058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:57.792396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3a14c169-ee02-4abd-8bae-be450c73183c', 'id': '58902dfa-8f68-4cae-a849-ef582736a197'} May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.794053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.795249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.796299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) May 23 16:02:57.797375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:57.798407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-474ab4ad-5c11-469e-b14b-23d866c47a62) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:57.798407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5c0936e7-c3ae-4ff5-a28f-cc18791f6682 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 400 len: 269 microversion: 2.1 time: 0.151765 May 23 16:02:57.798407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 677/1234] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 269 bytes in 153 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:57.798407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.798715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.805606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.806055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.806809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.807334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.820090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:02:57.827723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d12ac6b4-96b4-4f53-b0bb-7d5a359e612c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1561 microversion: 2.1 time: 0.152924 May 23 16:02:57.828371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 558/1235] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1561 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:02:57.833146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.833722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.836027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d713324e-9b49-476f-8bc8-62a27796eb73 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041862 May 23 16:02:57.836575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 678/1236] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:57.839394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.839912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.840735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.840955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:57.841312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.841462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:57.847703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.848164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.848936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:57.849375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:57.924987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.965683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:57.993599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:57.995822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:57.997154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:57.997998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:57.999023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.000259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.001253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) May 23 16:02:58.002322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.003405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-998500e5-e2ee-4ea0-91e7-93bb16dc43e0) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.003405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0ae8bb6-c059-4bef-8e8f-cea426e11c23 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 0.167856 May 23 16:02:58.003405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 559/1237] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 169 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.003956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.005122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.011859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.012393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.013051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.013649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.027688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:58.029942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.031663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.032412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.034218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.037245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.039519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) May 23 16:02:58.040956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.042499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a3067da0-0469-4a36-92cf-cbf7d49fad79) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.042499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b45af93-c902-469f-a491-c461ba8a6571 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.194016 May 23 16:02:58.042499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 679/1238] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:57 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 195 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.042499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.042861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.044034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f75460e4-72bf-4629-b43e-2f704c12a937 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045236 May 23 16:02:58.044577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 560/1239] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:58.045239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.045711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.046462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.046921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.049161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.049665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.055263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.055752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.056456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.056887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.126953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.195611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.196401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:58.199931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:58.202264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.205070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.207757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.209191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.210627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) May 23 16:02:58.212198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.213736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6fbabd9a-a8f2-41f5-a210-d426df4553ed) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.213736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c1927d20-b194-4d1d-af63-1553925b102d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.166953 May 23 16:02:58.213736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 680/1240] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 168 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.213736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.214109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.214945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.215443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.216360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.216796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.239121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 HTTP/1.1 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:58.241455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:58.242879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3a14c169-ee02-4abd-8bae-be450c73183c', 'id': '58902dfa-8f68-4cae-a849-ef582736a197'} May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.243539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.244566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.245681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.246705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) May 23 16:02:58.248007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.249234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0e90f0ab-f949-45f6-8620-a10cf553531c) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.249234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a217ce4-0a4e-491c-af5d-ac2be94600bd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 400 len: 269 microversion: 2.1 time: 0.197318 May 23 16:02:58.249234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 561/1241] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 269 bytes in 198 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.249234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.250994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.257165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.257628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.258397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.258836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.263620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:58.289450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2374f641-8c0f-41e7-92cb-0c4f51d9142f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044962 May 23 16:02:58.290019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 562/1242] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:58.294571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.295070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.304612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.304612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.304612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.304612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.353065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:02:58.360502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d3ceb53d-fad7-4473-93f9-3964420ce08a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1561 microversion: 2.1 time: 0.158475 May 23 16:02:58.361062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 681/1243] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1561 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:02:58.365688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.366184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.371730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.372181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.372968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.373407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.385236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.451631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.478966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 HTTP/1.1 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:58.481143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:02:58.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2', 'id': 'f022774e-0ec3-444b-af35-2dc0e23213e3'} May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.483659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.484701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.485846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.486912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) May 23 16:02:58.490107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.491400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0fb1b634-bb2f-46c2-9a6e-3cb7dfc74278) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.491400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c38e05ee-a772-4f13-a476-933e66dea361 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3" status: 400 len: 269 microversion: 2.1 time: 0.191604 May 23 16:02:58.491400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 563/1244] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments/f022774e-0ec3-444b-af35-2dc0e23213e3 => generated 269 bytes in 192 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.491400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.491762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.496740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.497202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.497975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.498413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.518621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:58.521347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.522664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.523488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.524535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.526962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.529367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) May 23 16:02:58.531503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.543407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2f4ca5a2-8822-437a-9d1f-3a683715ec75) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.543407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-81897257-d066-40d2-bfc4-5279ce220c08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.160769 May 23 16:02:58.543407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 682/1245] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 162 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.543407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.544593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.544755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ed3aa79-f5be-46d2-b40b-4dd6cdc1c583 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048827 May 23 16:02:58.544755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 564/1246] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:58.544755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.544755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.545350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.545350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.545350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.545350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.545944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.545944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.546485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.546929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.558927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:58.630797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'services' on Instance uuid 3e090a96-ef41-4548-ac75-e4dd2f371539 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.670515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lazy-loading 'services' on Instance uuid 3a14c169-ee02-4abd-8bae-be450c73183c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.698431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a HTTP/1.1 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:02:58.700697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:02:58.702209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'server_id': '3e090a96-ef41-4548-ac75-e4dd2f371539', 'id': '8ebaf0a3-1be2-4045-b12b-f4ad1873a26a'} May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.703201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.704412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.705792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.707076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) May 23 16:02:58.708331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:02:58.709549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-74a14f26-8b88-4c47-ba37-7f887edc1c4a) {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.709549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6ddea4bc-0d6b-4a9d-9cd4-2e71616d58c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a" status: 400 len: 269 microversion: 2.1 time: 0.176783 May 23 16:02:58.709549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 683/1247] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments/8ebaf0a3-1be2-4045-b12b-f4ad1873a26a => generated 269 bytes in 179 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.711926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.712591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.718549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.719076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.719741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.720172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.738655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Request method failure captured: May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 HTTP/1.1 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6x6fGctYnYEQj6NPmluLkkiPQHeufiS803_QFigtmeGinZ5g8S3T1yh10eudRv8xORJPUYhvAuHjYzbRhXAM6IOY05iztJnGwG4lFrj5C4Xm8s8VTu0RudFqy5ojbsiMea9tQi9DjAL-iqe2juQ-R-TL2-ZY8qCcTRuYhBTHhWuGhE May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:58.740897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7e22027c2e5948a6ba154c837a7673a9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7e22027c2e5948a6ba154c837a7673a9", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7e22027c2e5948a6ba154c837a7673a9 May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-577538905 May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: cc4c3405e93141fc851f22e49eee09bd May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-VolumesSnapshotTestJSON-577538905-project May 23 16:02:58.742447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '3a14c169-ee02-4abd-8bae-be450c73183c', 'id': '58902dfa-8f68-4cae-a849-ef582736a197'} May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.743206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.744397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.745503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.746483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) May 23 16:02:58.747659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.748945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2221a9d4-c6b3-42d2-9d17-25f7cdfed666) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.748945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d6bf6317-fa84-4c84-8d0e-125936b6226b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197" status: 400 len: 269 microversion: 2.1 time: 0.207267 May 23 16:02:58.748945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 565/1248] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c/os-volume_attachments/58902dfa-8f68-4cae-a849-ef582736a197 => generated 269 bytes in 208 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.748945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.749544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.756567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.758415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.761390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.761390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.792459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2c2782f7-bde2-4aa4-b7f3-07ca158f86a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048271 May 23 16:02:58.793020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 566/1249] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:58.797638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.798165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.805021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.805473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.806231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.806670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.863300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:02:58.871669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4041c757-2f11-4d13-9a60-65ddf4a58c46 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1561 microversion: 2.1 time: 0.163348 May 23 16:02:58.872240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 684/1250] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1561 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:02:58.879911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.880160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.890381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.890890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.891705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.892137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.896968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lazy-loading 'services' on Instance uuid 087911cb-6647-4769-a496-f0a0f2e0482c {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:02:58.925818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3dc10694-6d99-429b-808c-9247c74f4ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051686 May 23 16:02:58.926688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 685/1251] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:58.931462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:58.932698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:58.939330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.940169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.941590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:58.942028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:58.958264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Request method failure captured: May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 HTTP/1.1 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii60WJpKH7j1NNm3zMUicqIWoMKDN37yH1Yt3kg9LxOG1nIJybtyxtiTYtPqm2fYxNbII3be58xPh12EcX-UZx_Aepy2bxUmtKUk605muOBOQS3-cuL01pOnJpUh-d5aPwtwNHPljwoXtIB3atxrZy7AAVMWqHCcY2464FIh0fW9Boss May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:02:58.962217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/8ce8b36a28714d618becacd8e2925a75"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_8ce8b36a28714d618becacd8e2925a75", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 8ce8b36a28714d618becacd8e2925a75 May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-TestMinimumBasicScenario-2097968565 May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 16a2ff4a80da41bf84f19450000a9da1 May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-TestMinimumBasicScenario-2097968565-project May 23 16:02:58.963766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '087911cb-6647-4769-a496-f0a0f2e0482c', 'id': '15326cd2-64ce-47d8-90db-d8ae702d0251'} May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.964595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 23 16:02:58.965740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.967376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.968441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) May 23 16:02:58.969992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:02:58.971492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-018666dd-cf8c-4eb4-8383-6353cb484478) {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:02:58.971492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7991285d-51b9-4257-bfa6-6fe1bdcbc4e6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251" status: 400 len: 269 microversion: 2.1 time: 0.170490 May 23 16:02:58.971492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 567/1252] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:02:58 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c/os-volume_attachments/15326cd2-64ce-47d8-90db-d8ae702d0251 => generated 269 bytes in 172 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:02:58.989284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f645f716-412d-4417-a531-117bcf759943 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061384 May 23 16:02:58.993237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 686/1253] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:59.545415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:02:59.546405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:59.556597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.557056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.557838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.558285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.664602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0b65da90-285c-4157-b8b4-4d424c5ee9b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/action" status: 200 len: 41128 microversion: 2.1 time: 0.123478 May 23 16:02:59.665380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 568/1254] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 16:02:59 2022] POST /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c/action => generated 41128 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:02:59.752666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:59.753196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:59.757366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-4e18573a-41e0-4c7d-a04f-383d12136f33 req-5d459c5d-73c0-4796-816b-6167fcdae339 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:02:59.759482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.759932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.761172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.761704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.767291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-4e18573a-41e0-4c7d-a04f-383d12136f33 req-5d459c5d-73c0-4796-816b-6167fcdae339 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.774113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-4e18573a-41e0-4c7d-a04f-383d12136f33 req-5d459c5d-73c0-4796-816b-6167fcdae339 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.779771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-4e18573a-41e0-4c7d-a04f-383d12136f33 req-5d459c5d-73c0-4796-816b-6167fcdae339 service nova] Creating event network-changed:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 16:02:59.785629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-4e18573a-41e0-4c7d-a04f-383d12136f33 req-5d459c5d-73c0-4796-816b-6167fcdae339 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032050 May 23 16:02:59.786495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 569/1255] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:02:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:02:59.790948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:02:59.810935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:02:59.812239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:59.838272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.838762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.842149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.842149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.945925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-41979dd5-1f6f-47db-9a0c-fad5bdb1f049 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action" status: 200 len: 37845 microversion: 2.20 time: 0.145334 May 23 16:02:59.947355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 570/1256] 10.222.0.32 () {62 vars in 1318 bytes} [Mon May 23 16:02:59 2022] POST /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d/action => generated 37845 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 361 bytes (2 switches on core 0) May 23 16:02:59.951931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:02:59.952951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:02:59.961604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.968517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:02:59.968517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:02:59.968517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:00.006592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a82af32a-6cf5-4e71-829c-53dd4dac2527 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058600 May 23 16:03:00.007572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 571/1257] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:02:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:00.020455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:00.021429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:00.040355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:00.041073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:00.043266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:00.043266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:00.082120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-79b7b1df-d6bb-4227-82e8-8c8c6c995fa7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072693 May 23 16:03:00.084433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 572/1258] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:00.158624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-276bb131-1a55-4dd5-aef4-e7ba24446ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "DELETE /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 204 len: 0 microversion: 2.1 time: 0.409439 May 23 16:03:00.159418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 687/1259] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:02:59 2022] DELETE /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:03:00.167884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:00.168511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:00.176005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:00.176459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:00.177232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:00.177671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:00.398809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:00.406641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f3458817-21a9-439c-8ab9-9066ddf96b98 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1567 microversion: 2.1 time: 0.242667 May 23 16:03:00.407301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 573/1260] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1567 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:01.018697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:01.020500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:01.029610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.030136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.031305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.031618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.064269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-78f47d10-c8d2-4e73-ae89-b3dd32b3833e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049786 May 23 16:03:01.064648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 574/1261] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:01.070023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-5d42c0e4-5aad-4f19-82bd-46b2cdb2863a req-cd9c3743-be7e-4e21-bd7e-ba9aa6c05444 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "909fad86-0068-4016-bf26-7246f0b13e53", "name": "network-vif-deleted", "tag": "46b4b1bb-1870-4ede-9ef3-5032d3773430"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:03:01.082625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5d42c0e4-5aad-4f19-82bd-46b2cdb2863a req-cd9c3743-be7e-4e21-bd7e-ba9aa6c05444 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.083080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5d42c0e4-5aad-4f19-82bd-46b2cdb2863a req-cd9c3743-be7e-4e21-bd7e-ba9aa6c05444 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.096085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:01.097788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:01.098729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-5d42c0e4-5aad-4f19-82bd-46b2cdb2863a req-cd9c3743-be7e-4e21-bd7e-ba9aa6c05444 service nova] Creating event network-vif-deleted:46b4b1bb-1870-4ede-9ef3-5032d3773430 for instance 909fad86-0068-4016-bf26-7246f0b13e53 on hv-cinder-83588 May 23 16:03:01.104707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-5d42c0e4-5aad-4f19-82bd-46b2cdb2863a req-cd9c3743-be7e-4e21-bd7e-ba9aa6c05444 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102720 May 23 16:03:01.105692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 688/1262] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:03:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:01.107487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.108141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.109298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.109853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.145382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9207bff5-f63b-4056-a147-a231028ff052 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054108 May 23 16:03:01.146187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 575/1263] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:01.425306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:01.425928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:01.432018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.434323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.434323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:01.434323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:01.499005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:01.507005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32e601b3-b5d4-49b6-b20d-fbfef5f4e456 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086113 May 23 16:03:01.507592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 689/1264] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:02.075985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:02.076955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:02.083958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.084416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.085186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.085628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.115640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7056ddb1-9d0b-4949-b30f-318355c13c8a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043556 May 23 16:03:02.116200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 576/1265] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:02.156976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:02.157855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:02.164208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.164655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.165433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.165861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.193882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f3b296f8-5061-440c-9cd2-f1e7bae6a0b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040969 May 23 16:03:02.194422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 690/1266] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:02.529787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:02.530404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:02.536323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.536773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.537523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:02.537956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:02.596892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:02.604290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aab0797c-5782-4bde-adc7-596b30028897 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078347 May 23 16:03:02.604836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 577/1267] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:03.135583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:03.136794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:03.142878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.143358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.144116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.144570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.174114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad23fe46-9daf-4be7-938f-bfe7c430bcf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042449 May 23 16:03:03.174730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 691/1268] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:03.205499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:03.206547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:03.213549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.214000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.214761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.215227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.244866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6864968e-390e-4ec1-88bc-44f36d761bb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043301 May 23 16:03:03.245417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 578/1269] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:03.620975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:03.621591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:03.627795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.628251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.629016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:03.629457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:03.689532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:03.696963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a6f732e-e069-4ec3-ac6a-f2d53fd94eda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079906 May 23 16:03:03.697530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 692/1270] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:04.185299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:04.186199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:04.192513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.192961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.193725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.194166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.197969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:04.222972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-511e70da-0ca6-48c9-936d-c96aa62cd6b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041713 May 23 16:03:04.224202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 579/1271] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:04.256731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:04.257674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:04.264237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.264679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.265426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.265855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.294116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-def8fbe5-7efe-4412-b2fe-6145d8656b2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041893 May 23 16:03:04.294647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 693/1272] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:04.713640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:04.714263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:04.720749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.721204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.721966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:04.722407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:04.781945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:04.789467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aad13edc-23a6-4a64-9d63-f0aa728175f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079625 May 23 16:03:04.790234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 580/1273] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:05.234841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:05.235804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:05.244952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.245400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.246166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.246601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.275009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee5785e6-0c24-4490-bcde-f617cd2a74ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044290 May 23 16:03:05.275638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 694/1274] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:05.305520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:05.306446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:05.313190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.313742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.314497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.314932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.342754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cbece45c-8921-4ce9-ab1b-c374e5e896a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042029 May 23 16:03:05.343322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 581/1275] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:05.806103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:05.806722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:05.812866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.813313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.814062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:05.814491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:05.877331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:05.885177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db932140-7d51-489c-9755-5fa2340a0c74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082965 May 23 16:03:05.885733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 695/1276] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:06.286737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:06.287713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:06.293771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.294223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.294985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.295461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.324477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4538bcb9-37d0-4780-810f-59db0376701f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041749 May 23 16:03:06.325059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 582/1277] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:06.353745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:06.354666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:06.360605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.361053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.361802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.362230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.391108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74cc237e-c78e-4322-b633-c0726a582b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041324 May 23 16:03:06.391651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 696/1278] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:06.901945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:06.902555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:06.908642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.909089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.909839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:06.910272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:06.973756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:06.982227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a2a6d70-df71-42be-b513-5eeb195e454f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084067 May 23 16:03:06.982792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 583/1279] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:03:07.336684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:07.337610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:07.344111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:07.344556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:07.345298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:07.345727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:07.373490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9446d827-dfe4-4427-a10a-5f33ba950a43 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041432 May 23 16:03:07.374074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 697/1280] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:07.402742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:07.403743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:07.410247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:07.410716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:07.411494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:07.411931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:07.439969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-415a0866-e8fd-44ee-b7c9-624728d02fcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041189 May 23 16:03:07.440510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 584/1281] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:07.998745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:07.999381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:08.006025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.006475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.007250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.007687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.025624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:08.068494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:08.075768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73fae69f-a689-40c4-a7c7-fd840e0e50e8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080843 May 23 16:03:08.076330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 698/1282] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:08.384834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:08.385760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:08.392095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.392554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.393317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.393753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.422771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-14a4a22f-7285-4d93-ac4d-327adc3f2db9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041964 May 23 16:03:08.423388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 585/1283] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:08.451751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:08.452733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:08.460262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.460715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.461479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:08.461915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:08.491976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dc28c630-2198-4540-b496-cdc9d66b7df2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044173 May 23 16:03:08.492465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 699/1284] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:09.092332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:09.092947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:09.099581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.100032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.100792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.101229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.160709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:09.168585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-41348de4-b1d0-4a60-a945-3e2282af06a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080088 May 23 16:03:09.169149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 586/1285] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:09.434350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:09.435324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:09.441455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.441903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.442652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.443082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.471565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-801a08bc-55ba-4501-af04-0f48acfe4587 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041182 May 23 16:03:09.472467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 700/1286] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:09.502942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:09.503900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:09.511216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.511667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.512428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:09.512863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:09.541611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ee2eeef-5c2f-40a0-8f31-82f571248c46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042567 May 23 16:03:09.542163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 587/1287] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:10.185144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:10.185687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:10.191724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.192173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.192935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.193373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.258123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:10.267227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cb1db663-3c32-457c-8795-99e0dd0ed7e7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085877 May 23 16:03:10.268149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 701/1288] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:10.484050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:10.484986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:10.491178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.491634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.492395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.492831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.523204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de8bf44e-a098-49bd-afac-53a70e095e65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042990 May 23 16:03:10.523841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 588/1289] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:10.553021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:10.553992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:10.563246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.563708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.564479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:10.564944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:10.594454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d3de6cf2-cf95-4e25-a2f7-399a58dc5fef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045384 May 23 16:03:10.595015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 702/1290] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:11.288623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:11.289241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:11.295405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.295852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.296610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.297046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.356541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:11.363871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cc5ca304-c07e-4abe-bcd4-2793b8e937a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079682 May 23 16:03:11.364430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 589/1291] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:11.534342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:11.535308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:11.542030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.542487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.543290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.543739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.571790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-895b09ad-3183-428e-8ce0-edecbb1691df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041411 May 23 16:03:11.572417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 703/1292] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:11.606196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:11.607055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:11.613498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.613949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.614739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:11.615196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:11.643452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8d091613-26c4-4f46-97b2-22d05f257d36 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041250 May 23 16:03:11.643998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 590/1293] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:12.380990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:12.382196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:12.388519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.388970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.389731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.390171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.449355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:12.456655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8495f075-b9f5-4f28-a115-dad003925506 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079520 May 23 16:03:12.457237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 704/1294] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:12.585876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:12.586741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:12.593081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.593539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.594305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.594774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.624332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8183c55e-96a3-46ca-a5b9-0671485037be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042525 May 23 16:03:12.624937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 591/1295] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:12.655228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:12.656153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:12.662970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.663452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.664207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:12.664640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:12.692838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7ed5537e-df48-4e4f-9918-280e483c8a92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041517 May 23 16:03:12.693381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 705/1296] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:13.473153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:13.473779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:13.479995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.480453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.481251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.481697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.541633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:13.555214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3dc234e9-b18d-4c00-b042-f483435991fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085805 May 23 16:03:13.557539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 592/1297] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:13.636280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:13.637235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:13.643368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.643811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.644563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.644994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.672829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f27333a7-319c-43c0-bb0a-5a13dee4d23d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041399 May 23 16:03:13.673437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 706/1298] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:13.703903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:13.704936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:13.711413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.711867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.712622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:13.713058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:13.741455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-12289963-71c0-4538-97f0-0eb3fa1ff9b3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040943 May 23 16:03:13.741933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 593/1299] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:14.571142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:14.571667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:14.577638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.578086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.578871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.579339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.645229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:14.653221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7d522f34-2c53-46da-bdca-9a6f6003e22b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086087 May 23 16:03:14.653781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 707/1300] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:14.684485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:14.685409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:14.691418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.691866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.692620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.693053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.721941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-529faf65-e991-4ef1-ab96-ec4e615105f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041454 May 23 16:03:14.722524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 594/1301] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:14.752753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:14.753682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:14.760550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.761254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.762014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:14.762451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:14.790674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-63ef8a2e-caf7-4ae1-903a-a0c91f5bd41d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041813 May 23 16:03:14.791237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 708/1302] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:15.670661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:15.671310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:15.677304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.677751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.678560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.678991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.732515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:15.733451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:15.739409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.739573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:15.739979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.740620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.741058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.743268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:15.747520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-894d95c7-c7e0-443e-9b3a-5c22d392fa42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080683 May 23 16:03:15.748084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 595/1303] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:15.770371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b05aa22b-4bc9-445d-9e1a-1623239e67b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041778 May 23 16:03:15.770928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 709/1304] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:15.801936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:15.802858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:15.809509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.809956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.810709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:15.811170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:15.839752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83a63bdc-d45b-4c0b-8ee1-888aaf7a7428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041960 May 23 16:03:15.840287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 596/1305] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:16.764207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:16.764837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:16.771240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.771793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.772419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.773406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.781812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:16.782781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:16.786163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:16.790267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.790721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.791528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.791969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.820805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70210db2-584e-48fb-9b93-f889db2b0350 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042901 May 23 16:03:16.821577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 597/1306] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:16.834056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:16.842043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4957965c-6bbd-466d-abec-d29f3ceff80a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081959 May 23 16:03:16.842600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 710/1307] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:16.850386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:16.852070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:16.858340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.858794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.859592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:16.860032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:16.888044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-549b4243-b684-4319-a568-95520adba04e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041592 May 23 16:03:16.888585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 598/1308] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:17.833030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:17.833963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:17.841037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.841493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.842246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.842685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.859298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:17.859920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:17.866305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.866757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.867551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.867996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.872402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3ae28232-3b3e-4fd6-88bb-7870eee2e1ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042526 May 23 16:03:17.872402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 711/1309] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:17.899132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:17.900165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:17.906077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.906516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.907297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:17.907733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:17.928846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:17.936393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fd4b07a4-7187-44f3-a9fa-5fd785858d51 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041838 May 23 16:03:17.936563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4307029b-9888-459b-92c5-7a094786db4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080997 May 23 16:03:17.937149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 712/1311] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:17.937266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 599/1311] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:18.883548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:18.884511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:18.891449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.891911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.892692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.893130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.923272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-93dbac16-cf15-46a7-aeb7-cb403d05a393 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043700 May 23 16:03:18.923954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 713/1312] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:18.949843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:18.950658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:18.954217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:18.954838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:18.957365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.957826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.958595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.959274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.962683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.963163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.964620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:18.964620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:18.992058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d1acf2c-f7a2-4a34-a879-b570883e6583 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047934 May 23 16:03:18.992636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 600/1313] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:19.035724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:19.043883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33a28c44-bafd-4c9b-bea8-82ff92256e6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.093908 May 23 16:03:19.044442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 714/1314] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:03:19.934550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:19.935509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:19.941763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:19.942222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:19.942996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:19.943476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:19.972800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-225eb2cd-baff-475f-837f-098c09ef1d07 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042214 May 23 16:03:19.973467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 601/1315] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:20.003709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:20.004644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:20.011333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.011785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:20.012540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.012973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:20.016622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:20.042113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0930c1c4-ee1e-4677-adb4-009ef80faa8a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042360 May 23 16:03:20.042668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 715/1316] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:20.061056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:20.061703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:20.067737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.068198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:20.068962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.069406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:20.129479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:20.137455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7d16e5d6-a7e8-4d6b-b795-8a924a9a1454 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080277 May 23 16:03:20.138012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 602/1317] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:20.985249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:20.986176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:20.989251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:20.992790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.993246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:20.993996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:20.994432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:21.024878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6644cd1e-a904-43b4-8f53-11c5e35a9296 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043951 May 23 16:03:21.025545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 716/1318] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:21.053501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:21.054940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:21.061351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:21.061798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:21.062549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:21.062979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:21.091055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dfe40a36-385d-40bb-992b-ff76e5d6e5f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041440 May 23 16:03:21.091612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 603/1319] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:21.162233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:21.162860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:21.169414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:21.169867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:21.170630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:21.171069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:21.231349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:21.239168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c26cf5ac-d567-4b71-b567-8ccaef6a3149 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080624 May 23 16:03:21.239679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 717/1320] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:22.036596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:22.037520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:22.044180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.044625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.045371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.045802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.073643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f14e0c97-0821-4389-8e11-7115bef40d1d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041020 May 23 16:03:22.074227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 604/1321] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:22.102398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:22.103941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:22.110321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.110775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.111568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.112009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.141503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f997dea7-6c44-4580-b0bd-376b47598695 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043076 May 23 16:03:22.142054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 718/1322] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:22.256813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:22.257425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:22.264098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.264545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.265291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:22.265723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:22.325285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:22.333022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7c8ff4f-44f0-4bdb-9320-536ee79d0e03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080246 May 23 16:03:22.333589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 605/1323] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:23.085293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:23.086253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:23.092319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.092773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.093535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.093975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.123061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b7e74957-5254-4190-8117-7b70e69255af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041753 May 23 16:03:23.123682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 719/1324] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:23.152955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:23.153892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:23.160495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.160954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.161720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.162163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.192310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-38214035-5d9e-4496-ad76-65a7a90f07c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043267 May 23 16:03:23.192853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 606/1325] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:23.350208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:23.350759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:23.357945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.358389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.359164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:23.359597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:23.422120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:23.429533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c52eeed4-7206-4a7f-9fe5-d19a937ab966 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083229 May 23 16:03:23.430091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 720/1326] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:23.933421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:03:23.933577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:03:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:03:24.135326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:24.136515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:24.142515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.142970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.143773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.144212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.172297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4c5387ed-0175-406f-b888-584d4c2cb50e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041538 May 23 16:03:24.172880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 607/1327] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:24.203594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:24.204523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:24.211108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.211577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.212336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.212771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.246105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-65a81fbb-3bba-4bce-9e9a-201ea4283b4a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046330 May 23 16:03:24.246671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 721/1328] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:24.449731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:24.450260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:24.456230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.456680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.457434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:24.457877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:24.519466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:24.526983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e00bec2b-9b47-4add-8961-ec832733b47d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081203 May 23 16:03:24.527568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 608/1329] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:24.608774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:03:24.608925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:03:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:03:25.183835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:25.184679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:25.190503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.195465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.195465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.195465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.221238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cec53a0f-f3ea-4853-93de-7cbbb37495b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041473 May 23 16:03:25.221874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 722/1330] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:25.257774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:25.258720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:25.266005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.266460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.267246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.267683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.296530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0420290d-32cc-4c55-820b-414411642a68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042745 May 23 16:03:25.297072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 609/1331] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:25.544522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:25.545157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:25.551065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.551637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.552473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:25.552909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:25.612301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:25.619758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-71347c0d-9d03-4ba9-931b-e838346bb4bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079810 May 23 16:03:25.620312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 723/1332] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:26.232819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:26.233859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:26.239977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.240422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.241185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.241618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.270455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-353f9bab-d12e-4edc-a375-233953850d39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041653 May 23 16:03:26.271045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 610/1333] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:26.308489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:26.309409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:26.315773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.316310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.317061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.317493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.345762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9bc98304-706f-4dbb-80b1-f1af172e7c37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041705 May 23 16:03:26.346302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 724/1334] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:26.636340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:26.637479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:26.643502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.643959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.644719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:26.645156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:26.704202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:26.711538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef8d7e98-4aee-4b41-a897-2f2243cde86e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079341 May 23 16:03:26.712137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 611/1335] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:27.281915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:27.282845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:27.288833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.289285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.290677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.291142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.318667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0a2845b-12f1-40d8-93d3-2b7f4924a5d2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040725 May 23 16:03:27.319274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 725/1336] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:27.357771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:27.358766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:27.365251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.365703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.366459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.366897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.382977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:27.396879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ca9e137-abfc-42e8-8d25-6dad3c635420 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043015 May 23 16:03:27.397425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 612/1337] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:27.728900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:27.729425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:27.735399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.735845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.736598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:27.737035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:27.799848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:27.807232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-310d4353-62d3-47f3-a969-ce77f7d5cf28 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082768 May 23 16:03:27.807795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 726/1338] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:28.330239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:28.331207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:28.338850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.339333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.340124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.340568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.370268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-537717d8-dbca-4697-a6d3-8144b24ca3a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044009 May 23 16:03:28.370946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 613/1339] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:28.408500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:28.409432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:28.415964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.416418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.417182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.417624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.445413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db8376d2-d912-4b6c-b354-b7ebde2025cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040808 May 23 16:03:28.445955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 727/1340] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:28.825039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:28.825645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:28.831822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.832268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.833023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:28.833452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:28.893098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:28.900456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef043445-ebcf-4710-a29b-48ca7de94174 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079998 May 23 16:03:28.901008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 614/1341] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:29.381006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:29.381977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:29.388282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.388737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.389495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.389944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.419387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-566eb3b7-a59a-46b7-92a6-906492895164 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042348 May 23 16:03:29.419995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 728/1342] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:29.457375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:29.458296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:29.465365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.465815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.466569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.467005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.498067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da38eef0-60a5-4b8c-a13d-15c422335acf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044446 May 23 16:03:29.498541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 615/1343] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:29.917196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:29.917722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:29.923809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.924248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.925005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:29.925439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:29.984923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:29.992276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14766ad8-1c69-4b7b-b6b5-dfce9359588a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079050 May 23 16:03:29.992838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 729/1344] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:30.429899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:30.430843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:30.437507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:30.437968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:30.438733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:30.439202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:30.467516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e4ae214-5492-4791-b89b-fac8815daeda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041543 May 23 16:03:30.468112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 616/1345] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:30.508800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:30.509668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:30.516396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:30.516852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:30.517616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:30.518060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:30.547071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-40f5cbcd-39f6-44a3-b00d-daedcaa3b351 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042285 May 23 16:03:30.547640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 730/1346] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:31.009016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:31.009638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:31.028378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.028834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.029612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.030044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.095042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:31.102594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-098eb5b9-8eef-436b-bcfd-dcec242d6133 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.097411 May 23 16:03:31.103232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 617/1347] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:31.481855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:31.482822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:31.488931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.489378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.490145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.490628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.516342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:31.520713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33ff9d46-2a4b-4c09-a635-f4731353acf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045326 May 23 16:03:31.521277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 731/1348] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:31.559021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:31.559970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:31.566649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.567088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.567876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:31.568303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:31.597634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-209afe9f-6d8d-439d-b46d-782ffc4a4757 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043111 May 23 16:03:31.598169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 618/1349] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:32.121108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:32.121632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:32.127621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.128070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.128823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.129256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.191082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:32.198438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1bbdee09-13f8-40fa-8677-781a9c78b981 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082207 May 23 16:03:32.198992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 732/1350] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:32.532286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:32.533219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:32.539287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.539746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.540507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.540946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.569528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39be2bb4-4a9e-40ae-a2fe-6a6c50769004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041218 May 23 16:03:32.570222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 619/1351] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:32.609483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:32.610410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:32.616866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.617317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.618073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:32.618512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:32.647019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b8a6aed4-a980-4165-bf72-22f4e3dbb3bc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041420 May 23 16:03:32.647585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 733/1352] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:33.214592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:33.215238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:33.222281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.222839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.223721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.224155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.290193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:33.297902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5ba398ac-9295-4107-af22-90874fa3dc2f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087413 May 23 16:03:33.298459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 620/1353] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:33 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:03:33.580841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:33.581796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:33.588217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.588672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.589441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.589893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.621521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-732f0524-b7a7-4c58-8f5c-fc5573cf03b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044765 May 23 16:03:33.622226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 734/1354] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:33.659293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:33.660237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:33.667246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.667711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.668487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:33.668929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:33.698141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8316da82-b180-498a-85fe-baa880ce0d84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042822 May 23 16:03:33.698701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 621/1355] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:34.316323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:34.316941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:34.323509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.323951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.324698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.325126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.384022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:34.392167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d78fb708-a78b-42b1-ac72-d593bb6c10b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079654 May 23 16:03:34.392722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 735/1356] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:34.633054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:34.633896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:34.640819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.641269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.642028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.642465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.670591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ed38cb4f-8a54-4c5d-a339-9318bfc2cc91 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041561 May 23 16:03:34.671226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 622/1357] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:34.709656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:34.710582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:34.716949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.717437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.718190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:34.718623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:34.746639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2cb8c4c3-02ec-4036-9369-0e10fa138711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040909 May 23 16:03:34.747200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 736/1358] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:35.409031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:35.409681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:35.415679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.416282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.417248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.417686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.480284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:35.492857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a9ebe75-bd8e-4f2d-9ca0-8b1f8947cbf9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087022 May 23 16:03:35.493363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 623/1359] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:35.681491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:35.682977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:35.689215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.689682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.690431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.690865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.718988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc39ef1b-efa8-4d47-aafd-fed3d02c6686 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041414 May 23 16:03:35.719627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 737/1360] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:35.757944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:35.758813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:35.765566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.766021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.766791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:35.767258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:35.797077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bccc0967-db5e-45a8-a8f2-a7ffd1d74dca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043100 May 23 16:03:35.797627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 624/1361] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:36.512451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:36.512988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:36.519217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.519664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.520421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.520855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.586001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:36.594059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5cf0432c-c75b-41b2-ae34-741afb250789 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088042 May 23 16:03:36.594714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 738/1362] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:36.730054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:36.730983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:36.737128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.737628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.738429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.738871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.769200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ca32ad70-4f2e-488d-8a92-32cce18a7873 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043128 May 23 16:03:36.769801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 625/1363] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:36.810213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:36.811172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:36.817175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.817626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.818379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:36.818814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:36.850071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6bea9d75-f7b1-4d24-98c9-b3ab3fd2762b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045422 May 23 16:03:36.850637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 739/1364] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:37.611777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:37.612322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:37.618374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.618857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.620159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.620596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.679829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:37.687850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-86413ed3-bf17-4a7b-a533-8320f4c13ab4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080078 May 23 16:03:37.688419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 626/1365] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:37.780479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:37.781429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:37.787912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.788361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.789120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.789553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.817893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6d4bd67-8904-40a5-80f7-7ec054589265 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041447 May 23 16:03:37.818493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 740/1366] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:37.861027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:37.862008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:37.868370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.868824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.870133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:37.870564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:37.898560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f653b333-b39d-42d4-93ff-470bfbe075d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041413 May 23 16:03:37.899123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 627/1367] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:38.704844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:38.705384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:38.711398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.711846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.712601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.713057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.774131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:38.781600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32ff1d7c-d45a-4dc4-8513-dfa255fbf68b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080752 May 23 16:03:38.782169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 741/1368] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:38.830138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:38.831080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:38.837120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.837576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.838926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.839393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.868701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e00fcfa1-ceb8-4bc9-a769-f08a59e879a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042588 May 23 16:03:38.869342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 628/1369] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:38.910108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:38.911061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:38.917573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.918018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.918773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:38.919231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:38.948508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bfff083-5505-4847-ac43-f074d8fb9698 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042338 May 23 16:03:38.949056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 742/1370] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:39.801227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:39.801842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:39.810725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.811206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.811970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.812410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.816388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:39.878855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:39.882347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:39.883369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:39.887421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-50237407-fc12-4603-8b9e-fe130b59f205 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.090053 May 23 16:03:39.888003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 629/1371] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:39 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:39.890872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.891486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.892281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.892791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.922046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-819504ce-6292-4270-8667-55d94249dff4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043805 May 23 16:03:39.923382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 743/1372] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:39.960562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:39.961435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:39.967673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.968130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.968896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:39.969337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:39.999500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0201bff8-8bae-4926-aafa-5b054bc756f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043259 May 23 16:03:40.000064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 630/1373] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:40.904257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:40.904872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:40.910829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:40.911305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:40.912052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:40.912477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:40.933445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:40.934431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:40.940840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:40.941298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:40.942066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:40.942499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:40.972595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8d3fe0a7-ac3a-47e4-905b-dd40710b821d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042971 May 23 16:03:40.973345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 631/1374] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:40.976274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:40.983655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd00d502-9fcc-4adc-8ae6-30df688b9363 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083825 May 23 16:03:40.984197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 744/1375] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:41.011762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:41.012725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:41.018715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:41.019201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:41.019972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:41.020413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:41.048951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a3f6839a-9dc0-4351-a5a2-4abce14f4645 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041884 May 23 16:03:41.049496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 632/1376] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:41.983999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:41.984971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:41.991873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:41.992329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:41.993103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:41.993544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:42.000752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:42.001372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:42.009469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:42.010495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:42.011613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:42.012056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:42.024134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ece30d6-e1f5-40aa-b626-02ee8efb9709 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044002 May 23 16:03:42.024704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 745/1377] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:42.061050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:42.061973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:42.069810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:42.070261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:42.071025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:42.071485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:42.075949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:42.083544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-41a00b5b-af50-4cd6-acaf-c312c7323364 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086553 May 23 16:03:42.084120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 633/1378] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:03:42.105336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6dab9925-49c2-434e-8e48-d8794774d3bf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048566 May 23 16:03:42.105893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 746/1379] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:43.036066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:43.037025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:43.044132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.044575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.045332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.045764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.073869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fad32474-78a7-4ccf-b03c-a10be1970626 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042069 May 23 16:03:43.074413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 634/1380] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:43.099795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:43.100416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:43.107537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.107991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.108755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.109191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.116690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:43.117596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:43.123673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.124121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.124876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:43.125313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:43.155376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-28ada35f-b3d1-447e-8866-afca758bbe4c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042400 May 23 16:03:43.155940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 635/1381] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:43.173396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:43.181033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74dbffbe-6fdf-4732-a888-f7a5ac1ee5d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085086 May 23 16:03:43.181596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 747/1382] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:44.085399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:44.086346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:44.096763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.097218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.097981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.098415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.135931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a7cdc30a-6d36-4537-8cec-86c4292850cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053291 May 23 16:03:44.136785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 636/1383] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:44.166562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:44.167519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:44.173885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.174329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.175080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.175534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.191461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:44.198103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:44.198732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:44.204619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.205243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e07a5eeb-8cd1-4381-a3c0-9defcc08b650 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042173 May 23 16:03:44.205357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.205558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 748/1384] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:44.208865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:44.208865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:44.266512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:44.274110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4ad3da0-d606-4ac1-817a-9c1cd91d4350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080285 May 23 16:03:44.274671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 637/1385] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:03:45.148333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:45.149290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:45.156211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.156660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.157415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.157854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.186558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-92398077-1216-4217-a65f-331a4fba2307 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042744 May 23 16:03:45.187171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 749/1386] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:45.216670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:45.217596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:45.224048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.224498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.225241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.225678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.255406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b3fe1ae-9d2a-479f-b885-15c3833eabcf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043211 May 23 16:03:45.255959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 638/1387] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:45.290594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:45.291783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:45.297818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.298269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.299024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:45.299488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:45.361512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:45.369044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f78e8e15-8ef1-4d2f-93d8-b8c07f5da382 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082185 May 23 16:03:45.369599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 750/1388] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:46.198594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:46.199466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:46.205674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.206127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.206893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.207362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.236334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6ec1f177-eebd-45df-91e4-936dfb5926ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042280 May 23 16:03:46.237267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 639/1389] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:46.266874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:46.267902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:46.274215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.274682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.275992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.276424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.304173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eed6fc2d-3dc9-48e3-b3da-d485057a492b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041236 May 23 16:03:46.304723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 751/1390] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:46.385637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:46.386245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:46.392441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.392888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.393642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:46.394383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:46.454441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:46.461891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf93dac5-b893-4c6e-bfff-19555e358f65 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080502 May 23 16:03:46.462448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 640/1391] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:47.248361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:47.249286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:47.255280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.255730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.256494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.256928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.286796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5792334f-33ec-41e6-9d6b-eed0dd3f2b55 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042879 May 23 16:03:47.287415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 752/1392] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:47.315666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:47.316608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:47.318893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=87 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:47.324187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.324640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.325400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.325836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.353690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57d1cc0d-33a5-4653-878e-a00339da2ba6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041948 May 23 16:03:47.354316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 641/1393] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:47.481371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:47.481792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:47.487958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.488343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.489100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:47.489533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:47.556835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:47.564367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d82a2803-11d2-425e-b956-fd40a8d549ba tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087306 May 23 16:03:47.564931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 753/1394] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:48.298101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:48.299038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:48.305276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.305725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.306484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.306920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.334703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77002813-02a0-4549-9196-52b50c1b6149 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040557 May 23 16:03:48.335337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 642/1395] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:48.365274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:48.366261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:48.372631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.373091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.373846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.374282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.403432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ef5f156-25da-4473-a635-ba8b090990a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042047 May 23 16:03:48.403978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 754/1396] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:48.581646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:48.582189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:48.588647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.589091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.589841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:48.590273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:48.649429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:48.657351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d01e954-7f1d-4adb-9c10-be56c6f25d4c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079553 May 23 16:03:48.657940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 643/1397] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:49.346247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:49.347223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:49.353762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.354211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.354962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.355421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.383973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7c867e4a-7345-4981-8f06-274e004f6112 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041718 May 23 16:03:49.384597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 755/1398] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:49.414836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:49.416318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:49.422700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.423180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.423938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.424366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.452842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7d74e672-01be-4a56-9ad0-b76753ad4fbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041887 May 23 16:03:49.453383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 644/1399] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:49.674021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:49.674549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:49.680503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.680953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.681708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:49.682145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:49.744992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:49.752378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5750a52-501d-423a-a3c0-7b458a714584 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082329 May 23 16:03:49.753195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 756/1400] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:50.395627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:50.396869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:50.402857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.403350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.404114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.404549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.428403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:50.433288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1515ec04-1e40-406c-b7ee-a7bb1758f643 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041599 May 23 16:03:50.433837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 645/1401] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:50.464134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:50.464999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:50.471343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.471789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.472564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.473000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.501148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8f19204-9f56-4038-9aca-074da299d99b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040954 May 23 16:03:50.501694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 757/1402] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:50.768871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:50.769492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:50.775489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.775947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.776712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:50.777148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:50.836565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:50.844123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ee9cef02-efff-4dc5-beb8-39e6cb42b58e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079437 May 23 16:03:50.844677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 646/1403] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:50 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:51.445776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:51.446714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:51.449230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=83 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:51.454343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.454799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.455716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.456207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.486106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d86f3e90-472a-4d38-bb60-1591a61758ec tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044955 May 23 16:03:51.486895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 758/1404] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:51.513351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:51.514317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:51.520917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.521372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.522138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.522578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.551444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-224a76a5-477d-46fe-a0ec-c96c72330747 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042925 May 23 16:03:51.551996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 647/1405] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:51.861745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:51.862853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:51.868339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.869272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.870026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:51.870455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:51.933035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:51.940752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77eab4b4-a7c7-41d4-8ac7-f535c286e5be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083311 May 23 16:03:51.941321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 759/1406] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:52.498099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:52.498942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:52.504963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.505416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:52.506182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.506619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:52.535123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15e844bc-228d-4a9d-8d6d-6b3209847454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041597 May 23 16:03:52.535727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 648/1407] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:52.561658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:52.562590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:52.568986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.569442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:52.570198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.570633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:52.598875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c572fb5f-09c0-4384-9b28-ed1cbc44b3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041091 May 23 16:03:52.599457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 760/1408] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:52.957990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:52.958521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:52.965046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.965497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:52.966256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:52.966696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:53.026251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:53.033707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-22bbacb0-f8a9-4c74-8e4d-c2a34c150014 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079737 May 23 16:03:53.034261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 649/1409] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:53.548449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:53.549595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:53.555918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:53.556865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:53.557648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:53.558078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:53.593062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-13785710-5b4d-4419-8b5f-c6ef42f996f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050225 May 23 16:03:53.593883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 761/1410] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:53.611068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:53.612030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:53.618480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:53.619598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:53.620364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:53.620807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:53.651132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f9a4b4b3-7f00-4dd5-93b2-a4952b93347f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044393 May 23 16:03:53.651735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 650/1411] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:54.049952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:54.050572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:54.056511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.056958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.057708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.058135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.119946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:54.127537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d4c6347-a461-466e-9492-d3ddb047f085 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081398 May 23 16:03:54.128090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 762/1412] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:54.604635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:54.605477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:54.611653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.612219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.613192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.613630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.642317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4049a745-e06f-4064-89c2-2856a74ff3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041725 May 23 16:03:54.642847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 651/1413] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:54.663016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:54.663987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:54.670419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.670873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.671668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:54.672102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:54.700765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5bf57ed-e873-4c23-80cc-fc12e11dc1fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041699 May 23 16:03:54.701313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 763/1414] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:55.144038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:55.144657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:55.150567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.151012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.151810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.152246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.217348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:55.224819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72becec1-08da-40fe-b5cf-73637c32ef19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084879 May 23 16:03:55.228915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 652/1415] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:55.712022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:55.713439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:55.719357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.719809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.720561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.720988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.749446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e315eb7-59bd-4c91-a3f2-a086baf54145 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041228 May 23 16:03:55.749987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 653/1416] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:03:55.765945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:55.766869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:55.772839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.773278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.774024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:55.774452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:55.804573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-258f83c1-f2be-4bc2-9211-8721e9238ed0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.154792 May 23 16:03:55.805113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 764/1417] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:56.354704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:56.355351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:56.361404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.361853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.362612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.363049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.422813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:56.430119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83cb208f-6504-4f04-afc6-cf755bbc8ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.191193 May 23 16:03:56.430681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 654/1418] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:56.761763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:56.762803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:56.769120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.769687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.770303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.770730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.774260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:56.801230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b6dce76-c052-4290-99e0-b4af1d2f4df7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044201 May 23 16:03:56.801789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 765/1419] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:56.815253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:56.816246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:56.822750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.823266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.824033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:56.824471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:56.852649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a34e7f5c-b198-48b7-9111-7e658c9ed3a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041401 May 23 16:03:56.853194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 655/1420] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:57.447038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:57.447679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:57.454153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.454600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.455381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.455818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.515940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:57.523262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cc08a75a-c091-46c9-96f5-6c2c8dda952a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080072 May 23 16:03:57.523817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 766/1421] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:57.812167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:57.813010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:57.819638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.820225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.820838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.821278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.849674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10529699-8d06-48c1-adc9-cccdfaeabefe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041543 May 23 16:03:57.850217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 656/1422] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:57.864010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:57.864949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:57.871374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.871831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.872592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:57.873066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:57.901766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-906ed0c6-9b76-4f11-b42f-0cfe76124526 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041666 May 23 16:03:57.902311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 767/1423] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:58.540059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:58.540631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:58.546699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.547081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.547882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.548322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.609626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:58.617069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f460ae96-10fa-4eaa-a288-9ecf77a2a69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080965 May 23 16:03:58.617624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 657/1424] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:58.862115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:58.863135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:58.870397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.870981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.871638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.872077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.900543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e09c6ccb-ef1e-442a-88e3-6f3c96fa1e9d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042687 May 23 16:03:58.901110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 768/1425] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:58.913330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:58.914291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:58.921241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.921702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.922456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:58.922896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:58.951596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ea1e524-6120-4577-addd-09b78869adb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042302 May 23 16:03:58.952147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 658/1426] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:59.635117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:59.635927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:59.641959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.642403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:59.643182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.643619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:59.703500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:03:59.712226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-57cce6b6-4be1-4e82-a67d-74b81a0d489e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081866 May 23 16:03:59.713384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 769/1427] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:03:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:03:59.912716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:59.913567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:59.920581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.921079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:59.921911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.922349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:59.952437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf363fee-1135-4cfa-92df-36d41455799c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043770 May 23 16:03:59.952999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 659/1428] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:03:59.963707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:03:59.964677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:03:59.971315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.971766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:03:59.972534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:03:59.972976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:00.002216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a66cdc1-7e26-4cc6-b7c6-330eb1853879 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043132 May 23 16:04:00.002781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 770/1429] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:03:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:00.730004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:00.730613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:00.736649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:00.737132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:00.738155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:00.738590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:00.800040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:00.807382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2acb1ab4-06a6-4d81-a8ef-d8598e1b6337 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081852 May 23 16:04:00.807944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 660/1430] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:00.964392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:00.965265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:00.971655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:00.972104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:00.972850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:00.973278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:01.001437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9df0c1aa-3c94-4c19-85fb-02e64dd406c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041505 May 23 16:04:01.001982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 771/1431] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:01.014034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:01.015001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:01.021567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:01.022022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:01.022787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:01.023258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:01.052291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cfac5a58-f145-426c-8bfe-21437bbe3fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042610 May 23 16:04:01.052849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 661/1432] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:01.824134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:01.824753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:01.830794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:01.831279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:01.832033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:01.832467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:01.892550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:01.900046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-033958c5-74ed-4bb3-bd35-aa4857dbfedf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080292 May 23 16:04:01.900614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 772/1433] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:02.013182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:02.014120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:02.020624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.021083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.021851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.022328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.038372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:02.051148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53f454fc-7600-4571-880c-1e2f7d43afe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042204 May 23 16:04:02.051702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 662/1434] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:02.064252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:02.065171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:02.071545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.072005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.072756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.073188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.100708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f608ae8e-6ce2-407b-99de-56a017585db7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040535 May 23 16:04:02.101179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 773/1435] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:02.916482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:02.917120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:02.923609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.924054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.924810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:02.925243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:02.987734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:02.995705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3947a1d4-e45e-43e7-9b1a-c601b7f83730 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083095 May 23 16:04:02.996270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 663/1436] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:03.063426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:03.064357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:03.070813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:03.071302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:03.072692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:03.073130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:03.100873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9cadd1cf-35fc-4d36-abc4-06b22f27427d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041530 May 23 16:04:03.101422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 774/1437] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:03.111894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:03.112828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:03.119720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:03.120163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:03.120908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:03.121364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:03.150193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3cc53f2e-cb18-4c5a-876e-b0e0d52cfca3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042206 May 23 16:04:03.150736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 664/1438] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:04.012485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:04.013101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:04.019151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.019604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.020362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.020826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.080118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:04.088054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c87d83c1-fa09-4904-8483-958b9b6bac78 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079076 May 23 16:04:04.088676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 775/1439] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:04.112319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:04.113287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:04.119780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.120227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.120973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.121409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.150502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62afdac3-925c-4138-8d14-41f05d2d59fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041549 May 23 16:04:04.151034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 665/1440] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:04.161405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:04.162338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:04.168378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.168824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.169579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:04.170014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:04.199364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-971685b5-1e72-4860-90ff-5d11204036c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041748 May 23 16:04:04.199849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 776/1441] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:05.105384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:05.106004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:05.112169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.112633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.113428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.113836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.161982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:05.162818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:05.169914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.170366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.171148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.171617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.175229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:05.183350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1711f175-48ec-4578-8751-c7c2abbf2096 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082455 May 23 16:04:05.184061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 666/1442] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:05.200835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-44db21f0-9019-482c-b76f-3e43b921abd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042912 May 23 16:04:05.201374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 777/1443] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:05.210548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:05.211509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:05.217466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.217918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.218673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:05.219118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:05.248543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dfa00003-e865-4b8f-97cc-d24d3a96f004 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041826 May 23 16:04:05.249099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 667/1444] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:06.200906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:06.201432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:06.207750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.208282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.208994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.209427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.211944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:06.212872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:06.219217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.219666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.220820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.221257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.252868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7ddf7b1-9b6a-4f51-872c-ee95108c029e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044672 May 23 16:04:06.254734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 668/1445] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:06.261975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:06.262903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:06.268838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.269285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.270042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:06.270474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:06.270661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:06.278168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-21de1444-0ca8-402d-ab49-3db1ccbf038a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081760 May 23 16:04:06.278734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 778/1446] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:06.298549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a939166e-e621-47a3-a505-6ddc6dbfa3fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041244 May 23 16:04:06.299083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 669/1447] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:07.265229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:07.266191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:07.272624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.273079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.273835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.274269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.295068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:07.295740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:07.302071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.302522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.302952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8bb80f1-4c19-49d8-9273-5f004fafef1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041965 May 23 16:04:07.303338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.303557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 779/1448] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:07.304073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.309717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:07.310652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:07.316417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.316864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.317608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:07.318037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:07.347219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-85f21d03-b8e5-4420-9246-874dd3b2acaa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041270 May 23 16:04:07.347783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 780/1449] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:07.368007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:07.375564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cf584b27-67a8-4319-8c68-0cfecb3d9590 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084409 May 23 16:04:07.376128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 670/1450] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:04:08.314110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:08.315047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:08.321490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.321949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.322709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.323168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.347399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:08.351716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e0140ad6-4b15-4dc8-81b6-367d8d7a0a07 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041604 May 23 16:04:08.352276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 781/1451] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:08.358955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:08.368107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:08.368107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.368107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.368107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.368518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.393096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:08.393709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:08.396050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5648190f-bc44-4a09-adc7-169e98656bdc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040930 May 23 16:04:08.396599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 671/1452] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:08.399977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.400932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.401689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:08.402161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:08.461093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:08.468947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ddac5c6c-b398-4cb1-99c6-b9130f311953 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079959 May 23 16:04:08.469509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 782/1453] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:04:09.364026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:09.364946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:09.371992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.372441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.373195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.373708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.401457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fc512dcd-267e-42fb-b08c-384c62c89542 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041338 May 23 16:04:09.402007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 672/1454] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:09.407609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:09.408549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:09.414962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.415454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.416215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.416655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.446483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-29884614-1c0c-4a95-8582-0b018c502afe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042723 May 23 16:04:09.447031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 783/1455] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:09.486309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:09.486857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:09.492905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.493351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.494099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:09.494560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:09.555884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:09.564009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a6dad0d-c82c-446d-8e49-3dcda8590494 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081592 May 23 16:04:09.564589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 673/1456] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:10.413015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:10.413940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:10.420352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.420805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.421561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.422002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.450149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34cff710-6f81-4d6a-9acf-806867e6dd45 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041031 May 23 16:04:10.450727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 784/1457] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:10.459190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:10.461613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:10.469265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.469717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.470477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.470916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.499410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6449c5c3-9c47-42bc-9c41-4392f13995fd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045331 May 23 16:04:10.499891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 674/1458] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:10.580481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:10.581095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:10.587055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.587603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.588368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:10.588803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:10.648377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:10.655735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6bae9990-0ecb-46c0-be54-d51fb5dd4751 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079103 May 23 16:04:10.656288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 785/1459] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:11.461892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:11.462846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:11.469344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.469793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.470556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.470991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.500628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0760e0a3-9292-4278-a53a-258f6bf24d0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042570 May 23 16:04:11.501118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 675/1460] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:11.513220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:11.514136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:11.520169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.524566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.524566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.524566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.549964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6dda74cb-c025-4337-bff3-e3b0c63834e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042220 May 23 16:04:11.550516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 786/1461] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:11.672597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:11.673218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:11.679253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.679705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.680467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:11.680903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:11.742418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:11.750519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae67bf41-9c00-4c36-b14b-472c3564bd33 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081789 May 23 16:04:11.751166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 676/1462] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:12.512911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:12.513912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:12.520283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.520727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.521473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.521906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.550537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-41edb0f1-8046-4685-a57f-27e8db543e85 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042444 May 23 16:04:12.551073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 787/1463] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:12.560636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:12.561555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:12.567881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.568324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.569072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.569501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.599264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ea15d726-f5ab-468b-9a49-b2da3cb19b32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042410 May 23 16:04:12.599811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 677/1464] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:12.767750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:12.768306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:12.774125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.774573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.775359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:12.775803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:12.834767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:12.842109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ab89584d-3514-422a-8e5a-aae8a78acff6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078223 May 23 16:04:12.842671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 788/1465] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:13.562445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:13.563416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:13.569814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.570276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.571040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.571504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.600143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-123410fe-46cf-439f-b411-e63fb6af5a09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041731 May 23 16:04:13.600692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 678/1466] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:13.609817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:13.610981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:13.617263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.617741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.618484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.618914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.649096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cef1b9cd-15f6-4b18-86ab-b8d4c9ca3ede tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043099 May 23 16:04:13.649646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 789/1467] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:13.859260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:13.859795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:13.865864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.866308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.867058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:13.867519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:13.872021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:13.927004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:13.934954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e1d8a216-1d7d-4093-b7ae-5b22a5091109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079635 May 23 16:04:13.935536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 679/1468] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:14.611810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:14.612650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:14.619438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.619894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:14.620658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.621096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:14.649753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-143aa024-6dc7-4935-80fc-a14247d451b9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042001 May 23 16:04:14.650302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 790/1469] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:14.660608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:14.661531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:14.667900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.668357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:14.669108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.669542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:14.697077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a66e7e8d-c676-4226-a7b1-22220f2600b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040412 May 23 16:04:14.697607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 680/1470] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:14.951965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:14.952534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:14.958482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.960926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:14.960926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:14.960926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:15.021388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:15.030726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36e47334-c5af-4660-a245-c2b2e2ac0095 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083192 May 23 16:04:15.031329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 791/1471] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:15.661871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:15.662801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:15.669627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:15.670412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:15.671518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:15.672735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:15.703409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02116757-b99f-4177-b1ce-1233f18dfe0f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045488 May 23 16:04:15.703975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 681/1472] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:15.708989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:15.709926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:15.716331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:15.716786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:15.717550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:15.717991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:15.747428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ceaf973f-0887-48ba-943c-27cf5fefb579 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042174 May 23 16:04:15.747923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 792/1473] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:16.048134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:16.048685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:16.054817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.055354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.056109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.056570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.119335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:16.126860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3ecf2ab9-bafa-49ac-ae94-32827ad83b81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082700 May 23 16:04:16.127441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 682/1474] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:16.713902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:16.715440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:16.721942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.722387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.723160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.723594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.750761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-782d0548-2092-44d0-86ba-f8f07df633e3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040828 May 23 16:04:16.751329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 793/1475] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:16.758527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:16.759471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:16.766651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.767129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.768438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:16.769227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:16.800967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-37db83a5-8101-431d-a014-614b7cb405b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046178 May 23 16:04:16.801526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 683/1476] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:17.144194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:17.144840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:17.153997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.154775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.156120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.156857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.218756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:17.226276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-149ec1ff-2fc7-41ca-8c7f-e81449ac62fe tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085968 May 23 16:04:17.226834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 794/1477] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:17.762169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:17.763129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:17.769667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.770123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.770890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.771365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.801049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ad80c288-9a9b-46ff-aac5-6732b9a18dfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043103 May 23 16:04:17.801603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 684/1478] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:17.812359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:17.813330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:17.820013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.820795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.822129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:17.822900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:17.856231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0c62f4cc-e564-4657-a342-66a7c3e552c8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047722 May 23 16:04:17.856787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 795/1479] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:18.243732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:18.244343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:18.250976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.251494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.252304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.252735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.313295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:18.320675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-626f1e9a-308b-4fac-92e1-3540161c814e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080813 May 23 16:04:18.321234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 685/1480] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:18.813364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:18.814314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:18.820741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.821208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.821956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.822393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.867370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2344a7ce-e1a0-4d19-bc4a-e6d80697ec1a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057869 May 23 16:04:18.867944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 796/1481] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:18.869523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:18.870471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:18.872817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=90 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:18.878016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.878475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.879290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:18.879787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:18.911543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35177805-05c0-45a8-8619-d57bce19656d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048339 May 23 16:04:18.912097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 686/1482] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:19.336657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:19.337270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:19.343767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.344217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.344971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.345405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.405236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:19.412726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e24507c5-de31-43f0-a820-8747da6af45e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079999 May 23 16:04:19.413291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 797/1483] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:19.878906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:19.879896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:19.886372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.887078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.887849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.888281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.916426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-19368bb6-ef2e-494d-9eac-344222515826 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041553 May 23 16:04:19.916967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 687/1484] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:19.923361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:19.924321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:19.930720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.931208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.931966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:19.932402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:19.948612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:19.960863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c8fee678-144d-4c3b-89af-4d0e96c8950d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041507 May 23 16:04:19.964584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 798/1485] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:20.429627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:20.430240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:20.436465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.436906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:20.437653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.438080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:20.499313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:20.506706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-43323d83-8b2f-441b-a159-2fd9e13a83bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080970 May 23 16:04:20.507297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 688/1486] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:20.928565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:20.929439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:20.936863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.937319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:20.938078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.938518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:20.967006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-69ce9737-3ee3-4156-8e48-2624f441ee7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042529 May 23 16:04:20.967587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 799/1487] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:20.973837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:20.974755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:20.981917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.982366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:20.983179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:20.983615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:21.012812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03c5420d-5ad0-44cc-9651-8820c8ffacda tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042930 May 23 16:04:21.013350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 689/1488] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:21.524163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:21.524704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:21.530669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:21.531138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:21.531903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:21.532341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:21.592603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:21.600143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1eae61f2-984d-467f-94d8-4fde503fe59c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079922 May 23 16:04:21.600711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 800/1489] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:21.978814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:21.980272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:21.986722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:21.987205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:21.987969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:21.988406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:22.017287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62eb1382-4adf-4651-8aee-b9ddf548edf1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042743 May 23 16:04:22.017834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 690/1490] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:22.023453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:22.024388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:22.026677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:22.031365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:22.032158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:22.032934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:22.033362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:22.062682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bf91eeab-082d-4dba-8cee-b0f615105e3f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043109 May 23 16:04:22.063309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 801/1491] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:22.617097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:22.617667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:22.623813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:22.624267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:22.625060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:22.625497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:22.685827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:22.693417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c69dfb1a-a73e-48e6-8044-914145fd4a75 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080243 May 23 16:04:22.693989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 691/1492] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:23.029230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:23.030167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:23.037202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.037654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.038419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.038890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.067437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dbe5774d-2d7f-4fce-b5ac-2761dbdfbb93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042608 May 23 16:04:23.067990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 802/1493] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:23.073760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:23.074742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:23.081167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.081623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.082396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.082833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.111764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f3d70f01-28d5-4e02-835a-16f996dc789f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041866 May 23 16:04:23.112308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 692/1494] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:23.710061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:23.710668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:23.716696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.717140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.717889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:23.718318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:23.777486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:23.784773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b15ca00-1cd6-4bff-ba7e-4c14dd1a52d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078573 May 23 16:04:23.785323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 803/1495] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:24.078726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:24.079712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:24.086134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.086596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.087388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.087831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.116394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d9bb0408-4f3f-4599-b966-8d625d190b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041597 May 23 16:04:24.116945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 693/1496] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:24.124177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:24.125103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:24.131486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.131937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.132713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.133148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.162216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4cd539b1-6f3c-438f-afd0-fde9eeac3b73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042673 May 23 16:04:24.162754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 804/1497] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:24.801532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:24.802187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:24.808904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.809359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.810120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:24.810558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:24.871996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:24.879552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d32ca1f6-f6ba-45a4-96b0-86e3a92c0a1d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081747 May 23 16:04:24.880302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 694/1498] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:25.128166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:25.129065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:25.135503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.135989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.137488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.137917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.167664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5b08b876-e9d1-4f3f-8a5e-f1eb840bcd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043535 May 23 16:04:25.168218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 805/1499] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:25.174045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:25.174977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:25.180856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.181304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.182145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.182581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.206793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:25.211212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3f96773a-37c3-44a4-b545-9748b94cd95c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041485 May 23 16:04:25.217740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 695/1500] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:25.896865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:25.897479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:25.903683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.904133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.904890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:25.905326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:25.965231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:25.972746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9d58172a-161a-4e10-bcd9-c1e1d0e020c9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079865 May 23 16:04:25.973299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 806/1501] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:26.179070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:26.179966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:26.186846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.187315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:26.188066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.188498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:26.217240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b70ddd97-8a7f-4250-9ae9-c8850b68ee0d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042132 May 23 16:04:26.217781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 696/1502] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:26.226304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:26.227274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:26.233945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.234398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:26.235552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.235663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:26.265218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6de69f59-1d8b-46ce-9e0c-21b9e3ff02c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042747 May 23 16:04:26.265775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 807/1503] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:26.989233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:26.989849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:26.995823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.996272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:26.997025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:26.997459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:27.057189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:27.064625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b542d25b-fc94-487c-b865-8f5b6b863350 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079234 May 23 16:04:27.065179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 697/1504] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:27.228698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:27.229663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:27.237715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:27.238173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:27.238944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:27.239409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:27.268351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47517545-5909-44be-b35a-000433af2ec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043596 May 23 16:04:27.268940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 808/1505] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:27.276809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:27.278250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:27.284604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:27.285049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:27.285800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:27.286230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:27.314619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b63db245-6b24-4625-a1c2-160bbd8f1acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041746 May 23 16:04:27.315170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 698/1506] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:28.081270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:28.081884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:28.087945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.088401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.089155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.090209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.148960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:28.156938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-72bc97ef-875c-41c9-a697-97463e49360d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079617 May 23 16:04:28.157500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 809/1507] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:28.280423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:28.281352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:28.287736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.288191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.288942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.289372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.317241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-efd34fdb-12e5-4168-89b5-d2a1020388ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041402 May 23 16:04:28.317772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 699/1508] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:28.326273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:28.327237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:28.333657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.334107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.334866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:28.335331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:28.364179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e06a4a7d-dba4-46d9-817b-36662edcb96a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042362 May 23 16:04:28.364723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 810/1509] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:29.174009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:29.174622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:29.180639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.181094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.181847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.182283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.244785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:29.252310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b8a6f6e-d31f-40a8-bc76-efa35e80f280 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082137 May 23 16:04:29.252871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 700/1510] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:29.328488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:29.329423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:29.335923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.336379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.337144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.337584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.366367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f03dfc5d-ad77-4f1e-ae0b-4da9911d7b54 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041861 May 23 16:04:29.366916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 811/1511] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:29.374967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:29.375918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:29.382320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.382771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.383562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:29.383999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:29.412526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c185a6e9-4fb0-4e54-b19a-9da01bfeb002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041417 May 23 16:04:29.413064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 701/1512] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:30.269287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:30.269906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:30.276251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.276727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.278026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.278480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.337925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:30.345834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b03c256-16f4-4a55-9fd6-2a312d9cc214 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080545 May 23 16:04:30.346392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 812/1513] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:30.378652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:30.379750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:30.387312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.387759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.388526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.388983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.420440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5f0cc3f6-888d-4b30-a553-5e40f6655b9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046356 May 23 16:04:30.420992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 702/1514] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:30.424707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:30.425658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:30.432118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.432609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.433472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:30.434426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:30.462488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20d44eb1-da42-470d-9b1d-ada2ca5f67d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041934 May 23 16:04:30.463023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 813/1515] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:31.362428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:31.363036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:31.368956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.369408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.370156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.370586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.431184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:31.432052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:31.433130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:31.438813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c0f12eeb-2266-449a-94a6-366ba168bb0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080150 May 23 16:04:31.439402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 703/1516] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:31.443182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.443182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.443182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.443182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.471067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1528d745-bb54-41aa-bc4f-3d15da9d9dc0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043012 May 23 16:04:31.471677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 814/1517] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:31.474252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:31.475475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:31.482125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.482572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.483361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:31.483793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:31.513106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ed2af7ed-a2a3-4a96-b05a-534f571a3acd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042814 May 23 16:04:31.513578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 704/1518] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:32.455610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:32.456225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:32.462766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.463276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.464039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.464477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.468830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:32.483866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:32.484823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:32.491364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.491756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.492525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.492962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.524803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d6cd442a-76a5-4825-858c-b3d8c05b62a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046064 May 23 16:04:32.528562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 705/1519] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:32.528666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:32.531810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:32.534885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:32.535648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-66271654-31c0-46f1-b841-1f936b68b732 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083875 May 23 16:04:32.536216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 815/1520] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:32.538540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.538986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.539785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:32.540214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:32.569467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2acf947e-2632-44f4-856d-3ab75177635c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043028 May 23 16:04:32.570024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 706/1521] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:33.537328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:33.538187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:33.545259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.545714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.546474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.546921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.553149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:33.553755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:33.560711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.561161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.561916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.562348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.576110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fb3e514d-a8fa-4166-8620-d10bd10c9275 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042570 May 23 16:04:33.576513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 816/1522] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:33.581193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:33.582122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:33.588149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.588600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.589389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:33.589877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:33.625396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5d20d676-5497-4ea2-96a5-5a667771482a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047838 May 23 16:04:33.626149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 817/1523] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:33.636092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:33.644046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-41daa7f0-a06f-4af8-a915-c6769030ee6b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.094658 May 23 16:04:33.644605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 707/1524] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:33 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:04:34.587575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:34.588416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:34.595562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.596013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.596766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.598457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.629511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7c4d0a58-1595-4a39-87a1-13950db2828b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045950 May 23 16:04:34.630064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 818/1525] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:34.637177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:34.638142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:34.645059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.645507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.646269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.646705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.660932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:34.661581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:34.667648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.668101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.670810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:34.671916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:34.679387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24ce092f-dcd6-4b37-8263-3185dbe29a1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046228 May 23 16:04:34.679962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 708/1526] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:34.740198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:34.750111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b595219-cda0-49dd-bfc4-507b747658dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.093105 May 23 16:04:34.750686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 819/1527] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:04:35.642171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:35.643047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:35.649564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.650014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.650774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.651246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.679523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f0b5d28-c8b9-414c-9d82-01c30cca5311 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042152 May 23 16:04:35.680059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 709/1528] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:35.690814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:35.691773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:35.698118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.698568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.699357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.699793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.735856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-587c428f-6da7-4be3-8dd3-a79e9a792bb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048854 May 23 16:04:35.736408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 820/1529] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:35.767305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:35.767917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:35.773813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.774258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.775012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:35.775472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:35.834721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:35.842135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0f12678-4028-4af7-abf3-e4db922e8490 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078611 May 23 16:04:35.842693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 710/1530] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:36.691663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:36.692504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:36.698428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.698877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.699661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.700128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.728222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e1b87d23-f8f0-42a0-af5b-6d56380900d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040742 May 23 16:04:36.728765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 821/1531] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:36.747732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:36.748660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:36.755180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.755627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.756374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.756802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.785277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6f59e6c0-667f-45b6-9db1-da01463e0f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042048 May 23 16:04:36.785806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 711/1532] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:36.858599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:36.859234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:36.865231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.865677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.866434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:36.866868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:36.927084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:36.934560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b14c9d3b-36eb-42bc-829f-0de3a7c64c42 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079855 May 23 16:04:36.935148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 822/1533] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:37.739471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:37.740425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:37.746410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.746856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:37.747644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.748078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:37.764259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:37.776095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0beb1016-369d-4cea-b296-11c9deb322b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040739 May 23 16:04:37.776635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 712/1534] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:37.797323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:37.798284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:37.804853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.805301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:37.806059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.806497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:37.835712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47d74afa-a5ac-45eb-8828-8a631d41e254 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042970 May 23 16:04:37.836259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 823/1535] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:37.952509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:37.953130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:37.963062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.963541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:37.964317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:37.964755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:38.027610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:38.035616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9a3281c-4f09-42d8-afe8-bb80514dd69b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088108 May 23 16:04:38.036179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 713/1536] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:38.792936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:38.794008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:38.799996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:38.800577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:38.801188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:38.801623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:38.830915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-07990f31-7919-4af6-ab79-0d3817f38e3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041914 May 23 16:04:38.831494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 824/1537] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:38.847729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:38.848689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:38.855152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:38.855610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:38.856393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:38.856835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:38.884700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de587fa3-0e6e-442e-87ab-40bde233fb73 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040871 May 23 16:04:38.885236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 714/1538] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:39.052608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:39.053151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:39.059123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.060083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.060841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.061280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.120191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:39.128144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f3be1c15-98b7-4fc2-8bf6-446f4b8f6a4f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079542 May 23 16:04:39.128707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 825/1539] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:39 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:39.842565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:39.843512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:39.849498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.856937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.856937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.856937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.880079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73c1a285-ae24-4b0a-97d6-c357e2ab6eb9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042052 May 23 16:04:39.880633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 715/1540] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:39.896836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:39.897777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:39.904862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.905318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.906090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:39.906529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:39.936187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8fe18bae-b73f-4aac-9436-afa11b1e174f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043263 May 23 16:04:39.936737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 826/1541] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:40.145089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:40.145627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:40.151546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.151997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.152746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.153209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.212316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:40.219657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a1cdc91f-042e-4624-87a8-7b857796273e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078538 May 23 16:04:40.220211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 716/1542] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:40.892358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:40.893308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:40.899258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.899804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.900455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.900890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.930397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eb6c856a-e2da-4b6b-871a-aff21cd305e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042416 May 23 16:04:40.930877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 827/1543] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:40.947541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:40.948466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:40.955460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.955911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.956662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:40.957095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:40.985497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02ed6909-7c03-4bb5-9b1d-1652cb49719b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041837 May 23 16:04:40.986028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 717/1544] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:41.236335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:41.236949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:41.242863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:41.243345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:41.244096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:41.244525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:41.305046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:41.313037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-616fe2e6-3466-47c9-915e-b0cef912befc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080646 May 23 16:04:41.314063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 828/1545] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:41.941981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:41.942903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:41.948883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:41.949545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:41.950075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:41.950508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:41.979330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52e0f015-3948-42cf-924f-bad519a5140e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041254 May 23 16:04:41.980320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 718/1546] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:41.996881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:41.997855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:42.004312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:42.004768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:42.005534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:42.005973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:42.035379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a33e4e4c-e21d-4bae-ab5f-97bdecf57f4a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042448 May 23 16:04:42.035937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 829/1547] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:42.330217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:42.330832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:42.340110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:42.340562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:42.344431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:42.344926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:42.409378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:42.417031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-76a0b369-bc14-4476-a743-586be1f39cb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.090630 May 23 16:04:42.417666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 719/1548] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:42.993095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:42.994021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:42.999997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.000577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.001187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.001623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.030451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e3d14825-a574-4425-b752-cf8ff343bb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041346 May 23 16:04:43.030992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 830/1549] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:43.046312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:43.047272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:43.053680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.054126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.054879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.055339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.083297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f9c46efe-14b9-49ca-9348-94a63e4ac7da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041504 May 23 16:04:43.083834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 720/1550] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:43.434964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:43.435609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:43.441557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.442008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.442789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:43.443260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:43.502253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:43.505570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:43.509844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0cdb910e-4234-4c57-b2e2-878e7d36412d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079254 May 23 16:04:43.510406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 831/1551] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:44.042097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:44.043031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:44.049032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.049597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.050227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.050664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.079392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a1844cf6-76b2-4c44-85db-77c02b2b93c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041251 May 23 16:04:44.079946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 721/1552] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:44.094699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:44.095673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:44.102198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.102653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.103446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.103890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.131867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-35963590-6819-48f1-adc7-26cc8519a440 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042109 May 23 16:04:44.132414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 832/1553] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:44.527596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:44.528635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:44.534946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.535435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.536209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:44.536653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:44.597082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:44.604458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af06b73e-90ae-4e96-933a-8bf224f1ec10 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080772 May 23 16:04:44.605025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 722/1554] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:45.091018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:45.091958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:45.097952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.098527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.099149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.099580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.129070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1421d623-3038-4121-9372-d3b88709ba62 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042426 May 23 16:04:45.129658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 833/1555] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:45.143631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:45.144660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:45.151349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.151943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.152741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.153683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.182428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24dbc936-44a1-4801-96dd-1e92e2def90b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042774 May 23 16:04:45.182982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 723/1556] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:45.621245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:45.622438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:45.628511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.628958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.629705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:45.630138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:45.690063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:45.697434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fdfd18e6-e08f-4a8b-82cb-8b690fb9239e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080033 May 23 16:04:45.697992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 834/1557] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:46.141359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:46.142317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:46.149072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.149632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.150286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.150726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.179938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-04c3441c-2bdc-4680-b2b0-f41f241c4b21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043383 May 23 16:04:46.180483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 724/1558] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:46.193691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:46.194556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:46.201770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.202216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.203021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.203480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.232886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e35b5de4-76d9-4d9b-bb0b-37fb6dcefb49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043035 May 23 16:04:46.233453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 835/1559] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:46.714689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:46.715346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:46.721278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.721723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.722467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:46.722900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:46.784536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:46.791886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a4d94ff-067e-4e65-9400-a7194cc48fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081020 May 23 16:04:46.792425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 725/1560] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:47.191591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:47.192519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:47.199358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.199918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.200549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.200977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.229820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a6fd095-3420-4c09-bd89-7a433549b947 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042166 May 23 16:04:47.230372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 836/1561] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:47.244183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:47.245145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:47.251623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.252074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.252837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.253271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.281996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b2711078-ef9c-4ab7-94d1-323285c9d827 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041724 May 23 16:04:47.282539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 726/1562] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:47.809127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:47.810356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:47.816732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.817189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.817948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:47.818390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:47.877966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:47.885317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-882b250a-0104-4afe-82f4-9249a26ff9a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080135 May 23 16:04:47.885882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 837/1563] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:48.241883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:48.242839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:48.249423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.249984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.250620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.251056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.279651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d95d0931-f5b8-4f74-b418-450fab3949f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041717 May 23 16:04:48.280207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 727/1564] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:48.293392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:48.294963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:48.301325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.301772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.302516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.302954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.331210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f28c64c8-1be9-4351-8a44-906c55f0331b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042130 May 23 16:04:48.331748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 838/1565] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:48.901451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:48.902202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:48.908335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.908789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.909554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:48.909998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:48.969768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:48.977364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c6c1ce9d-c5a8-4ff6-9e9e-8127a395d228 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079754 May 23 16:04:48.980453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 728/1566] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:49.290848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:49.291829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:49.297979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:49.298548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:49.299209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:49.299654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:49.329929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d65822d-b08c-4a2e-ba95-5c112f72e16e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042984 May 23 16:04:49.332325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 839/1567] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:49.342709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:49.343674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:49.346224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=86 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:49.350981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:49.351465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:49.352220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:49.352650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:49.356892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:49.386790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-69c0cad1-a08d-4bf4-a64c-0396168bfa4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048893 May 23 16:04:49.387638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 729/1568] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:49.994117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:49.994731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:50.000934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.001374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.007752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.007752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.063285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:50.070658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-448119c0-f5b7-4616-9ecf-203a90c8b7d9 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080750 May 23 16:04:50.071234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 840/1569] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:50.343833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:50.344760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:50.351331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.351792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.352565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.353005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.381934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3d899425-d434-4029-8330-0f3465b726c1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042541 May 23 16:04:50.382500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 730/1570] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:50.397705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:50.398640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:50.405069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.405520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.406285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:50.406728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:50.435087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8cbe5cf1-4bb4-4aae-a7f9-e015f0aa9c4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040807 May 23 16:04:50.435651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 841/1571] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:51.087318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:51.087939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:51.094026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.094483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.095283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.095728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.157741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:51.165175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e9b55a2e-1f9c-42f3-b03e-1c1519c7cdaf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081776 May 23 16:04:51.165770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 731/1572] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:51.394276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:51.395204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:51.401179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.401734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.402367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.402801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.432073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d374275c-4b70-4114-b8e7-7060c2ba1034 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041560 May 23 16:04:51.432367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 842/1573] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:51.446304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:51.447357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:51.454236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.454689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.455486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:51.455921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:51.484775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ef14e5b-3822-4577-8300-e6968b4d49a7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042381 May 23 16:04:51.485323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 732/1574] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:52.182303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:52.182901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:52.189470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.189914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.190664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.191113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.250676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:52.258480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-00841ee8-2de9-4c87-816c-4919153b3d81 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080031 May 23 16:04:52.259055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 843/1575] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:52.443381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:52.444336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:52.450824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.451492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.452082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.452518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.481045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08c5e674-559e-426e-b51c-59e47c9f0336 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041595 May 23 16:04:52.482188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 733/1576] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:52.496461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:52.497359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:52.503740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.504191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.504946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:52.505381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:52.533817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a080430f-7d80-4afc-8bc5-6ba11b752cdd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041422 May 23 16:04:52.534352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 844/1577] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:53.275969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:53.276593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:53.282628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.283084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.283874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.284314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.342803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:53.350211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b9f5ba2-de69-441f-8740-fe7c52da16b5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078102 May 23 16:04:53.350769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 734/1578] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:53.492310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:53.493162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:53.495044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=88 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:53.500333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.500899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.501568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.502008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.531274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-843990d9-5985-442f-b657-9e64e86a51b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042931 May 23 16:04:53.531820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 845/1579] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:53.545713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:53.546684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:53.553120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.553573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.554337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:53.554806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:53.583574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c012e568-1f2d-4aae-b049-de0d2f426750 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042210 May 23 16:04:53.584113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 735/1580] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:54.368587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:54.369208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:54.375454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.375908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.376667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.377106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.436204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:54.443576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b959418-94b0-48e8-b2e9-f5a0c0c2428c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078790 May 23 16:04:54.444139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 846/1581] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:54.542709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:54.543601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:54.549604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.550208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.550828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.551302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.580368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b8396395-84ff-4f48-b732-c528c48cd04f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041603 May 23 16:04:54.580927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 736/1582] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:54.595569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:54.596533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:54.602922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.603415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.604179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:54.604618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:54.634218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca801fdc-93fa-4f51-be03-fdd3d52f2d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042506 May 23 16:04:54.634769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 847/1583] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:55.460478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:55.461101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:55.467659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.468115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.468882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.469323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.530925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:55.538382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5796581e-134a-4667-a55e-9c84d2d92d5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081911 May 23 16:04:55.538942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 737/1584] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:55.592678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:55.593607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:55.600005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.600576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.601207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.601642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.618317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:55.631521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5d97de0-5019-46d9-acfe-885274a6619c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043145 May 23 16:04:55.633394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 848/1585] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:55.645688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:55.646611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:55.652649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.653143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.653896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:55.654331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:55.682354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c36fd6ed-c135-46fa-aab3-590b7e0f737e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040510 May 23 16:04:55.682902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 738/1586] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:56.555916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:56.556466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:56.562765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.563278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.564038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.564475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.624041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:56.631697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5d45ec44-e221-4a89-a8b7-cbe0432ed2ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079766 May 23 16:04:56.632268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 849/1587] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:56.643456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:56.644377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:56.650664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.651244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.651885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.652316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.680444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ab666dd8-eef0-4c05-b28e-4b49b2f4c7a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041070 May 23 16:04:56.680983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 739/1588] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:56.693678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:56.694645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:56.701179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.701633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.702395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:56.702838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:56.731358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee2fdd8c-32d0-4677-ab5a-6456569ff7c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041532 May 23 16:04:56.731904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 850/1589] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:57.648918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:57.649449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:57.656067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.656625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.657252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.657677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.691505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:57.692431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:57.698903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.699388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.700157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.700589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.718366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:57.727665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5daec609-1280-48ac-b668-412367b00120 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082604 May 23 16:04:57.728246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 740/1590] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:04:57.730450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aec69647-825d-4944-b928-9ee8a259ff95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042904 May 23 16:04:57.731031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 851/1591] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:57.741820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:57.742813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:57.748770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.749225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.749993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:57.750444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:57.780630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5031270b-6420-4e8c-ab7a-ea82475a5ee6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042418 May 23 16:04:57.781520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 741/1592] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:57.982008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:04:57.982156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:04:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:04:58.742367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:58.743327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:58.744743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:58.745372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:58.750521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.750991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.751778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.752204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.752419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.752771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.753536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.753974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.781504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3d9fbdfa-8f97-4c04-a675-fd8b12c5db34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043102 May 23 16:04:58.782069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 852/1593] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:58.790951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:58.791891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:58.798288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.798734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.799509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:58.799944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:58.815807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:58.823345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dfd8331c-4719-4f38-96ab-091177e93d25 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082345 May 23 16:04:58.823909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 742/1594] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:04:58.829660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-98f1207d-780f-4239-8163-c42628a54bee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042316 May 23 16:04:58.830200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 853/1595] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:04:59.792758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:59.795782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:59.804314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.805248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.805530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.806096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.835047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3ca40569-d225-4e73-b658-cef4b117f091 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046292 May 23 16:04:59.838793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 743/1596] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:59.840950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:59.841568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:59.843984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:04:59.844913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:04:59.848543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.848996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.849765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.850269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.850709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.851206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.854947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:04:59.854947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:04:59.882034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef1da3ea-6180-4348-9009-48c10ed79bf8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042156 May 23 16:04:59.885504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 744/1597] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:04:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:04:59.915119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:04:59.923084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-706762ee-8a23-4672-9402-811fb31ff124 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086574 May 23 16:04:59.923659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 854/1598] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:04:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:00.846914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:00.848012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:00.854436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.854889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:00.855682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.856735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:00.880401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:00.884745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe795636-f05e-4d30-9fd4-976cfe375ac2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041816 May 23 16:05:00.889794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 745/1599] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:00.893919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:00.894840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:00.901372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.901881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:00.902627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.903580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:00.931735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6c9d7f1b-1133-4a86-b786-f43daa566897 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041847 May 23 16:05:00.932267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 855/1600] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:00.939573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:00.940231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:00.946513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.946967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:00.947766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:00.948210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:01.007537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:01.015272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e53c8b85-39cb-42a2-a4b7-3e555048b3b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079554 May 23 16:05:01.015837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 746/1601] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:01.896680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:01.897539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:01.904760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:01.905219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:01.906033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:01.906460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:01.934359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02b77596-1648-4472-b251-8fdc5dd57463 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041774 May 23 16:05:01.934904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 856/1602] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:01.942944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:01.943908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:01.950854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:01.951330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:01.952092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:01.952530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:01.981373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21eb0654-9ac2-4978-87bd-3bcbff71b2e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042274 May 23 16:05:01.981924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 747/1603] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:02.034386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:02.035227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:02.049168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:02.050552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:02.051347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:02.051782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:02.110998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:02.118297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67d64a36-f8ef-4d12-99b5-4196e52e13f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.089139 May 23 16:05:02.118882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 857/1604] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:02.946076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:02.947029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:02.954472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:02.954915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:02.955700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:02.956131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:02.983920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5eddeaa-1b78-4ae1-bf32-ff761f0e31f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042083 May 23 16:05:02.984464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 748/1605] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:02.992921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:02.993843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:03.000782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:03.001231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:03.002018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:03.002457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:03.033345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9a7adfdf-09ec-4614-a551-fe7c5d66c07b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044316 May 23 16:05:03.033907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 858/1606] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:03.135569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:03.136190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:03.142542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:03.142994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:03.143787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:03.144233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:03.204122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:03.211524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-27548e02-0438-4d7d-9434-6bd12a8968be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079737 May 23 16:05:03.212085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 749/1607] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:03.995338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:03.996270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:04.003182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.003633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.004393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.004826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.033218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5b7e2782-be31-4a3c-a985-e3cbf9fc4b3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041833 May 23 16:05:04.033759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 859/1608] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:04.045223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:04.046153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:04.052994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.053447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.054221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.056515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.088682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0afc5075-a8dd-482a-b258-4cf312e23e46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047834 May 23 16:05:04.089243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 750/1609] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:04.228151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:04.228726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:04.234869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.235360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.236128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:04.236565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:04.299800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:04.307282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-64c8c432-92e6-45fd-b5fd-15bf9192c8f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083063 May 23 16:05:04.307851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 860/1610] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:05.045179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:05.046148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:05.052293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.055306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.055306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.055306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.088758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a6a3eea0-54dc-4fc3-8b60-00623790bf55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047911 May 23 16:05:05.089309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 751/1611] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:05.100152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:05.101086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:05.108056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.108517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.109284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.109724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.139961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14c38d5e-6f65-474e-bfd3-c193fb1dc247 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043749 May 23 16:05:05.140431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 861/1612] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:05.324036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:05.324681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:05.330655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.331115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.331896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:05.332334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:05.392070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:05.399584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7cca010-d632-4ab5-a5b3-38404f729d74 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079384 May 23 16:05:05.400155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 752/1613] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:06.100444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:06.101375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:06.107992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.108447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.109242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.109677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.137811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-58a42945-14bd-4151-839d-b002eaecce2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041331 May 23 16:05:06.138369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 862/1614] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:06.151367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:06.152308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:06.159256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.159713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.160484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.160916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.191227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5ebc5e16-5ffa-41cc-a3c5-45e12d7ccc62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043745 May 23 16:05:06.191777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 753/1615] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:06.416900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:06.417508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:06.424153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.424597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.425344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:06.425771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:06.485130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:06.492494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb29fc22-c260-41aa-9195-56c8414cb641 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079381 May 23 16:05:06.492990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 863/1616] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:07.148888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:07.149848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:07.156903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.157468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.158096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.158530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.186365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-843a9beb-b2c8-4fa7-b9b0-163c444a602b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041391 May 23 16:05:07.186907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 754/1617] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:07.202877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:07.203903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:07.210309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.210752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.211530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.211962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.240887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1fbd9b5-e6c4-4420-885e-627fec665dcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042402 May 23 16:05:07.241441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 864/1618] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:07.509193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:07.509764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:07.516574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.517025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.517783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:07.518219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:07.577762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:07.585148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-63d8136e-5adf-4484-977b-61aa877564be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079921 May 23 16:05:07.585701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 755/1619] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:08.197780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:08.198687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:08.204594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.205186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.205775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.206205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.210211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:08.235773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c60c13af-4ba4-42b8-8726-c3d78ec301f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041547 May 23 16:05:08.236329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 865/1620] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:08.252569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:08.253496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:08.260007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.260451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.261203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.261634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.290846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4f6ef2dc-b513-4f18-ad71-fc792e37653e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042145 May 23 16:05:08.291421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 756/1621] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:08.602079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:08.602617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:08.609179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.609623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.610368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:08.610825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:08.672234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:08.679550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b2a2852c-c5df-413e-8dc9-e7de3b655011 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081690 May 23 16:05:08.680102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 866/1622] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:09.245877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:09.246843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:09.252859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.253410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.254058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.254494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.283738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e946d800-a1d0-4a33-8d75-887d946b8079 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041816 May 23 16:05:09.284283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 757/1623] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:09.301788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:09.302749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:09.309329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.309787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.310554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.310986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.339964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6136e252-c958-4f98-9df4-649913d9caef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042131 May 23 16:05:09.340511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 867/1624] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:09.697191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:09.697711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:09.703679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.704134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.704922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:09.705361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:09.999315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:10.007087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-26e94a23-914a-4f82-b0df-ffec7e07e341 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.313836 May 23 16:05:10.007683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 758/1625] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:10.295969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:10.296961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:10.302994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:10.303581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:10.304237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:10.304679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:10.333580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2889a251-be4a-4252-a3c8-ab6f36b6e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042229 May 23 16:05:10.334128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 868/1626] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:10.352343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:10.353990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:10.362090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:10.362537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:10.363337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:10.363779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:10.393276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7f425146-ff8f-450f-b5dc-d6fb28ca74fc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045206 May 23 16:05:10.393818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 759/1627] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:11.025102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:11.025720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:11.031820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.032271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.033033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.033475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.091903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:11.100015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83a3ddb6-6b4e-4f7c-9551-d55fdca89e0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078762 May 23 16:05:11.100583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 869/1628] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:11.349785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:11.349785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:11.353524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.354091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.354837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.355371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.385514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-43f42241-b56e-499c-bb17-e2faa8dd140d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044033 May 23 16:05:11.386051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 760/1629] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:11.404844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:11.405777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:11.412769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.413219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.413976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:11.414418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:11.444498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e8d9f38-e16d-4103-a92a-9438130de714 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043576 May 23 16:05:11.448896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 870/1630] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:12.116716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:12.117289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:12.123263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.123711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.124458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.125445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.183743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:12.191586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21c8971d-cf83-4122-85ef-e69dcae963a0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078810 May 23 16:05:12.192144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 761/1631] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:12.397564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:12.398461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:12.404431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.404884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.405649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.406697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.435789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f442146e-ab9a-4744-ad84-77c6cab7f28a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042220 May 23 16:05:12.436347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 871/1632] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:12.456252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:12.457198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:12.463588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.464043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.464809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:12.465254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:12.482240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:12.494525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0735b723-31cc-4c9c-8fdf-fbb4165c84e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042673 May 23 16:05:12.495081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 762/1633] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:13.208752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:13.209276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:13.215321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.215766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.216510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.216941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.277274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:13.285310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3460680d-bf31-4ad7-b463-5fa159e77b0c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080478 May 23 16:05:13.285870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 872/1634] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:13.447899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:13.448821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:13.456063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.456356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.457115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.457549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.490839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f73f76f6-62a3-4507-80ee-7b286a8d1113 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048008 May 23 16:05:13.492092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 763/1635] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:13.506464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:13.507409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:13.514155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.514602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.516040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:13.516477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:13.547081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b04ea2e-3b8e-4a1c-9e17-78c6a01af63d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044016 May 23 16:05:13.547657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 873/1636] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:14.301768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:14.302380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:14.308350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.308796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.309545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.309978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.371119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:14.378949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-956a4b96-5bfa-480a-a78a-2b54090322b2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081056 May 23 16:05:14.379546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 764/1637] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:14.503214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:14.504069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:14.510095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.510697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.511343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.511784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.541644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c8daf26e-aae6-4270-82e3-3b6c236f2075 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042506 May 23 16:05:14.542204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 874/1638] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:14.557732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:14.558674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:14.566496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.566959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.567777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:14.568283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:14.598529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-edc7f36e-b7b7-4050-b6d0-257e0a5e419c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044717 May 23 16:05:14.599079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 765/1639] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:15.395940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:15.396492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:15.402867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.403361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.404124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.404567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.463721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:15.471544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-71a15884-4a4e-458b-bbdb-de77a484dcc1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079496 May 23 16:05:15.472164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 875/1640] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:15.553613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:15.554568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:15.560574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.561140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.561783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.562798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.593065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b7c0c177-7a74-4c55-b704-750b1ee780db tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043406 May 23 16:05:15.594167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 766/1641] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:15.610382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:15.611349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:15.617659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.618110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.618865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:15.619333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:15.647494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db10c5ee-ef7b-46ba-87d4-11e08ec714f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041526 May 23 16:05:15.647969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 876/1642] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:15.948632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:05:15.948806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:05:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:05:16.489018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:16.489565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:16.495574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.496026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.496782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.497217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.556296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:16.563787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-303e747a-8b3a-486b-93f9-9f3e493d62ea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079284 May 23 16:05:16.564346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 767/1643] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:16.605094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:16.606027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:16.613217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.613796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.614427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.614867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.643550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-001e77ff-fa63-4df7-8efb-980b8757fb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042341 May 23 16:05:16.644101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 877/1644] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:16.658990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:16.659947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:16.666329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.666778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.667568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:16.668002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:16.696463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7dff544-fd59-4253-87a5-9414abbb47ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041887 May 23 16:05:16.697006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 768/1645] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:17.580785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:17.581328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:17.587341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.587795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.588555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.588992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.649245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:17.655278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:17.656199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:17.656822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74eb18fc-f52c-40c7-8c21-ed0272eab3b4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079974 May 23 16:05:17.657396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 878/1646] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:17.662048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.662497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.663292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.663725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.694118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d00ac3c1-f5ff-4d08-9504-93041dd0561d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042704 May 23 16:05:17.694681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 769/1647] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:17.708235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:17.709158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:17.715025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.715508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.716257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:17.716688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:17.744743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7c7c078-3519-4949-9758-698d3ee7cd9c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040846 May 23 16:05:17.745274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 879/1648] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:18.673458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:18.674013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:18.680537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.680980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.681759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.682187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.706392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:18.707339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:18.713737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.714190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.715034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.715569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.743876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9629b726-cee9-4a18-b74b-50b99889b69c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042125 May 23 16:05:18.744438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 880/1649] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:18.748287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:18.756452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:18.756618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3ba3e0b8-1068-4243-8058-a6bbc954111f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086951 May 23 16:05:18.757067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 770/1650] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:18.757440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:18.763691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.764135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.764895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:18.765333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:18.793335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:18.796132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b04073fc-408e-4562-92f5-75a58548f66e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043442 May 23 16:05:18.796702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 881/1651] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:19.755919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:19.757193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:19.759342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=84 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:19.764091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.764541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.765300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.765736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.773327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:19.773949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:19.783001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.783477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.784249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.784686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.795537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-082c3a9b-9589-4074-9d30-0415a175dbac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043445 May 23 16:05:19.796346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 771/1652] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:19.820639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:19.821658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:19.827855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.828302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.829159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:19.829594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:19.855127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:19.862540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b10bfb83-716e-4c99-a836-b31bb4cc3ad5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046066 May 23 16:05:19.863126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 772/1653] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:19.863378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c40fedf5-6a0b-405f-8e58-0709e9fe30b8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.093880 May 23 16:05:19.863933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 882/1654] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:05:20.808961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:20.810180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:20.817724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.818167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.818924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.819494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.848144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a991bf1d-2fee-454c-a5cc-d0657192e199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044033 May 23 16:05:20.848698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 773/1655] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:20.873732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:20.874747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:20.879753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:20.880376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:20.881755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.882211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.882983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.883438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.886197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.886670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.887469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:20.888159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:20.912303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e9a15f7-41f3-44b9-a4b6-0dea545e0d13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042451 May 23 16:05:20.912867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 883/1656] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:20.947411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:20.955384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e15c6e6e-b489-4ef5-b758-99eb7b1597d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079903 May 23 16:05:20.955940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 774/1657] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:05:21.861011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:21.861940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:21.867991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.868438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:21.869195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.869635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:21.897798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2788394-e479-4653-9cf5-8963b426631f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041945 May 23 16:05:21.898521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 884/1658] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:21.924382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:21.925342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:21.931781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.932230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:21.932985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.933419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:21.963298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b4ad8792-23cf-45b3-94ef-449389c02acf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043424 May 23 16:05:21.963842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 775/1659] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:21.972439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:21.973049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:21.978954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.979689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:21.980179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:21.980612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:22.039749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:22.046977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5c868d1f-c096-4a5b-b441-447ca3565f3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078954 May 23 16:05:22.047559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 885/1660] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:22.909058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:22.909992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:22.916074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:22.916536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:22.917299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:22.917739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:22.946811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0f540a29-435a-44b1-aec8-4486cd75e7c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041767 May 23 16:05:22.947431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 776/1661] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:22.973851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:22.974775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:22.981293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:22.981746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:22.982525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:22.982959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:23.016706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1281905-262b-4068-88a5-d8465f0891b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046671 May 23 16:05:23.017294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 886/1662] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:23.064710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:23.065466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:23.072235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:23.072693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:23.073480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:23.073917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:23.139604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:23.147111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-11bf92aa-2937-46f4-bc24-45700d3b1709 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087153 May 23 16:05:23.147686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 777/1663] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:23.958371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:23.959319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:23.965302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:23.965752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:23.966502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:23.966936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:23.995435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a9f0508-dc94-4c0e-bc89-f2703bc1ff59 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040948 May 23 16:05:23.995968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 887/1664] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:24.028308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:24.029244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:24.036439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:24.036894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:24.037654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:24.038093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:24.066122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aebc7c27-961a-4e45-8612-c5bd750cbebd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041743 May 23 16:05:24.066667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 778/1665] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:24.165755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:24.166398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:24.176875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:24.177328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:24.178259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:24.178694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:24.238696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:24.246283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f2fe9d7-b896-460c-8ecd-88cf70b1b5f2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086383 May 23 16:05:24.246845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 888/1666] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:25.005973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:25.006899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:25.013203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.013652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.014412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.014856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.018499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:25.044471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d3ece0c1-8969-409c-bda7-39ef01d79e7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042378 May 23 16:05:25.047214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 779/1667] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:25.076753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:25.077680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:25.080006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=88 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:25.084682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.085128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.085878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.086308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.115734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c05ee081-d9b9-4b2d-9b5c-eb4cff9c596b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042930 May 23 16:05:25.116327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 889/1668] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:25.263600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:25.264221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:25.271124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.271578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.272336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:25.272771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:25.333265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:25.340719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c102a023-8a08-434a-8280-75edd4248d4b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081609 May 23 16:05:25.341278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 780/1669] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:26.056494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:26.057927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:26.066083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.066528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.067303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.067736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.095620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6ecc59b2-76d1-40c7-8077-531a0b15d562 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043430 May 23 16:05:26.096225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 890/1670] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:26.126981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:26.127931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:26.134255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.134702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.135502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.135947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.164994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55ab4cd8-955d-4c4d-95cc-333ab411c9b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041899 May 23 16:05:26.165537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 781/1671] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:26.357443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:26.358133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:26.364058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.364511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.365257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:26.365694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:26.425770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:26.433336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1cd33c03-790b-491a-9709-c6e2e77cd50b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079803 May 23 16:05:26.433960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 891/1672] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:27.106976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:27.107926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:27.114395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.114835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.115629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.116059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.144636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e118fc30-de68-4299-9101-d336066d392c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041568 May 23 16:05:27.145298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 782/1673] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:27.176939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:27.177861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:27.184244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.184692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.185447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.185883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.214275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6f18700-d1ef-4c2c-82b6-c81179594a94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041816 May 23 16:05:27.214821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 892/1674] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:27.449566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:27.450141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:27.456134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.456577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.457327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:27.457758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:27.519080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:27.526405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-05855e18-1cd1-47ff-aa96-027ecfb6f835 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080754 May 23 16:05:27.526982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 783/1675] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:28.156330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:28.157242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:28.163234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.163687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.164441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.164876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.194016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce03d2e3-42ae-413b-a707-78a1d91a7ada tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041703 May 23 16:05:28.194620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 893/1676] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:28.225527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:28.226481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:28.232922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.233371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.234139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.234571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.263425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48bdaf2f-f8e5-4c19-bb8f-a57dc77ae22a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041443 May 23 16:05:28.263935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 784/1677] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:28.544691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:28.545225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:28.551661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.552110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.552862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:28.553297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:28.612229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:28.619543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c922679a-9c86-4741-bc07-0004339ff70f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079458 May 23 16:05:28.620104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 894/1678] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:29.205987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:29.206934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:29.213091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.213545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.214297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.214728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.244023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-983afad7-5c1b-4cd8-83f1-69af30dd0877 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042434 May 23 16:05:29.244617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 785/1679] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:29.274846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:29.275808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:29.282699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.283179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.283944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.284382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.312326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-21860466-49b0-4da1-9ef9-610b4174509f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041444 May 23 16:05:29.312872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 895/1680] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:29.637563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:29.638443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:29.645074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.645524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.646283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:29.646721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:29.706650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:29.714125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9db91159-5d85-411e-a1a7-f08449ce8ee4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080429 May 23 16:05:29.714682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 786/1681] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:30.256206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:30.257169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:30.263486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.263938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.264693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.265125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.293523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-58f421c8-ef0f-4a0d-a84c-0b5f73f95250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041276 May 23 16:05:30.294116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 896/1682] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:30.324083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:30.325008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:30.332414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.332861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.333612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.334075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.362394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2b4f16b9-aa9e-41ae-adc5-5ee3bb762cbf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042241 May 23 16:05:30.362932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 787/1683] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:30.731406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:30.732026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:30.737964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.738412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.739212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:30.739646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:30.799399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:30.806760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6047e5e0-e58f-4341-874e-926c8628c97a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079389 May 23 16:05:30.807354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 897/1684] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:31.305141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:31.306071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:31.312129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.312583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.313344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.314282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.343204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4027bb5-3c13-41d3-84ee-ee2ca4ce6cc7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041949 May 23 16:05:31.343784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 788/1685] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:31.374156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:31.378430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:31.382030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.382490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.383320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.383761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.400263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:31.412370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f9b3bc18-0ec7-40ab-ad80-f7bb3e1bb03a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042134 May 23 16:05:31.412922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 898/1686] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:31.823727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:31.824592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:31.831149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.831607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.832373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:31.832807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:31.898680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:31.906210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c28e4dd-5ef3-46b0-b329-e761ae42a1f3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086349 May 23 16:05:31.906775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 789/1687] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:32.354035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:32.354953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:32.361525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.361981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.362742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.363216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.393364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-65bafc93-31f8-42f6-a7f5-57c862c6a284 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043375 May 23 16:05:32.393959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 899/1688] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:32.423215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:32.430647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:32.430842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.431327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.432079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.432507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.461731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-328b7bf4-9ef5-4e30-904f-8e569e8da8e6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042587 May 23 16:05:32.462270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 790/1689] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:32.924024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:32.924684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:32.930658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.931122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.931900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:32.932338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:32.991940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:32.999276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96b6c66c-961b-4636-be3d-4164191444af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079632 May 23 16:05:32.999841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 900/1690] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:33.404967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:33.405897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:33.412465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:33.412914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:33.413670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:33.414109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:33.442620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5236ebdf-2c42-483a-99e9-c37e56479b9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041684 May 23 16:05:33.443244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 791/1691] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:33.473048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:33.473984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:33.480466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:33.480914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:33.481674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:33.482105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:33.510492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5053be4a-f5bf-4d1b-a3a6-8857c455b710 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041891 May 23 16:05:33.510987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 901/1692] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:34.017559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:34.018166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:34.024133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.024580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.025333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.025765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.085427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:34.093033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8f48bfdd-0884-4666-83db-c8667d685df2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080101 May 23 16:05:34.093590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 792/1693] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:34.454260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:34.455144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:34.461236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.461690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.462450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.462896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.492013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b925a0d0-19f5-42e8-8b29-637a3595d395 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041842 May 23 16:05:34.492627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 902/1694] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:34.522014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:34.522974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:34.529499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.529950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.530711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:34.531175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:34.565233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b05a9b0-176f-45df-9f60-741a7b648566 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047022 May 23 16:05:34.565805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 793/1695] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:35.109592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:35.110209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:35.116376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.116817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.117617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.118049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.178232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:35.186090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1be8535e-db3c-4702-bb0e-398fdb673289 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080441 May 23 16:05:35.186658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 903/1696] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:35.504073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:35.505070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:35.511078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.511560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.512319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.513010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.542387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b3b71f34-5cae-4727-b99a-1a11d2ced6fa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042775 May 23 16:05:35.543090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 794/1697] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:35.577214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:35.578144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:35.585159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.585705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.586463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:35.586896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:35.618351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-75b3d3db-fbe6-481e-bc40-06d98cae0fe0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045080 May 23 16:05:35.618908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 904/1698] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:36.204445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:36.205069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:36.211934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.212378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.213147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.213581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.280559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:36.284463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:36.288779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3029fb89-a280-4821-afd5-838ee326adf1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088721 May 23 16:05:36.289343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 795/1699] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:36.554209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:36.555049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:36.561038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.561485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.562237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.562667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.591383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-41edd221-6ab1-4dc4-ab61-c2866e4b155e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041299 May 23 16:05:36.591983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 905/1700] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:36.630067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:36.631002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:36.637440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.638158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.638923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:36.639385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:36.668226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4add40a8-b18e-4c41-8499-796f7e1838cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042475 May 23 16:05:36.668770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 796/1701] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:37.311742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:37.312358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:37.318819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.319308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.320065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.320494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.380231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:37.387723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c110e022-bdba-4eb7-bfbf-201883d84723 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079762 May 23 16:05:37.388549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 906/1702] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:37.603459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:37.604317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:37.610893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.611389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.612155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.612589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.641907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e48c5d01-b543-4510-84c9-370747b0dcdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042505 May 23 16:05:37.642538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 797/1703] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:37.679684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:37.680611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:37.686867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.687344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.688096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:37.689120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:37.716753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ac72df6f-9f0d-4de2-9f5a-6ff7ec631d6d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040939 May 23 16:05:37.717284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 907/1704] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:38.404632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:38.405194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:38.411789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.412245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.413011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.413453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.473602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:38.481090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a05721c-31ce-4e24-a025-63f3c5225e7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080396 May 23 16:05:38.481685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 798/1705] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:38.653226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:38.654142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:38.660155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.660551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.661292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.661719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.690210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6aac58dd-dd87-4b37-a262-52889ed2e276 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040905 May 23 16:05:38.690800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 908/1706] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:38.728698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:38.729632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:38.736001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.736457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.737222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:38.737660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:38.767537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9bae42f4-7f44-44f7-b6dd-0a25957c4779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042351 May 23 16:05:38.768086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 799/1707] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:39.498074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:39.498667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:39.504632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.505081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.505841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.506281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.565815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:39.573173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31ea1b8c-5ad5-4bbd-9623-c77aff781827 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079098 May 23 16:05:39.573725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 909/1708] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:39 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:05:39.702097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:39.703023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:39.709639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.710088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.710849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.711318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.740827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8ba6ede8-e737-4391-bbbc-62461425fdbf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042646 May 23 16:05:39.741430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 800/1709] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:39.778979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:39.779975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:39.786993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.787482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.788238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:39.788671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:39.817529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78298a7a-f0f0-4cb2-971a-5b6a3a91ec4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042453 May 23 16:05:39.818132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 910/1710] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:40.590547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:40.591114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:40.597140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.597596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.598388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.598832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.661009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:40.668631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b7aa7ccc-75f3-461f-b72d-ab2a5e3239e6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082030 May 23 16:05:40.669189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 801/1711] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:40.752213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:40.753127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:40.759136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.764995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.764995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.764995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.789223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-60e2a9e0-35a9-4a9d-9030-768b3b572359 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040938 May 23 16:05:40.789820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 911/1712] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:40.829150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:40.830080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:40.836553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.836997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.837758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:40.838173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:40.867551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-09e948b5-ba6f-4ad9-815e-0a8ec1d4acfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042448 May 23 16:05:40.868025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 802/1713] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:41.685336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:41.685899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:41.692472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.692913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.693656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.694082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.753858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:41.761270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-784e2c3e-e6ab-41df-a89e-3a8fd0977156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079841 May 23 16:05:41.761863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 912/1714] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:41.800500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:41.801429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:41.807378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.807833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.808595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.809037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.838125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3c3333f6-55c4-40ae-87d5-ef9a0d472eaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041524 May 23 16:05:41.838698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 803/1715] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:41.879632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:41.880683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:41.887179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.887630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.888645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:41.889081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:41.921616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f376c21-0232-4555-ac98-ce875ac25819 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046443 May 23 16:05:41.922202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 913/1716] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:42.778734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:42.779403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:42.785458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.785912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.786706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.787177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.846198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:42.849663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:42.850592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:42.853771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89c2e20b-82b1-49c5-9542-4d37049fb872 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078833 May 23 16:05:42.854329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 804/1717] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:42.856654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.857109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.857929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.858315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.887790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-25bbf7fc-0200-4be1-9bc5-427174659fa1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041961 May 23 16:05:42.888686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 914/1718] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:42.933746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:42.934673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:42.941264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.941714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.942473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:42.942906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:42.971373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a464ac9-6e69-4e2f-a2b7-e42aac5a6e60 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041417 May 23 16:05:42.971913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 805/1719] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:43.871114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:43.871667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:43.878176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.878627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:43.879427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.879865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:43.883756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:43.899569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:43.900502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:43.906882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.907379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:43.908153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.908594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:43.940972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57b81f82-3d7d-4e29-bbff-1e1436c82e9b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045300 May 23 16:05:43.941553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 806/1720] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:43.946082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:43.953600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8f4d4365-1011-401a-87c4-4bd24ee4dac5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086528 May 23 16:05:43.954804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 915/1721] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:43.982064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:43.982993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:43.989036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.989528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:43.990398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:43.990730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:44.020123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bef43afc-6670-45ee-8976-2d2d769f6af0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042023 May 23 16:05:44.020674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 807/1722] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:44.952193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:44.953119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:44.959520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:44.963502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:44.963502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:44.963502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:44.970883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:44.971524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:44.978032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:44.978481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:44.979279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:44.979715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:44.990912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7753c32b-6d90-472a-bb99-4b6185107d8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042831 May 23 16:05:44.992277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 916/1723] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:45.031829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:45.032911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:45.038987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:45.039560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:45.040313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:45.040745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:45.044245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:45.052438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e89317d-2058-40a0-9306-83b420e22400 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085401 May 23 16:05:45.053014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 808/1724] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:05:45.070676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-63be3c1a-f601-46df-9fd9-e6f0c4a89a0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042811 May 23 16:05:45.071248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 917/1725] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:46.001927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:46.002864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:46.009433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.009885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.010648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.011085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.039746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2013c417-a87a-4509-a03b-899dd9274d9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041749 May 23 16:05:46.040300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 809/1726] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:46.069292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:46.069896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:46.076266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.076715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.077460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.077888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.081599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:46.082556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:46.088414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.088870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.089649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:46.090089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:46.120051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-78a61a09-1c18-4e71-87c6-91ef45e05443 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042182 May 23 16:05:46.120617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 810/1727] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:46.138285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:46.146618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28d055f8-ef51-45b1-89da-b7f151fb3eb1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081163 May 23 16:05:46.147193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 918/1728] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:47.052098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:47.053026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:47.059649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.060097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.060862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.061332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.089678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4ed04b80-c763-468a-a5db-ff7eec2300ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042239 May 23 16:05:47.090208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 811/1729] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:47.131585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:47.132546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:47.139198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.139647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.140409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.140843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.163712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:47.164332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:47.169686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-58bba64a-86df-4841-a5e4-3988f27d49b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042026 May 23 16:05:47.170084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.170394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 919/1730] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:47.171369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.172173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:47.173125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:47.190728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:47.232847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:47.240830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b653d6b-a318-493b-a8a1-cec5c701cf71 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080821 May 23 16:05:47.241384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 812/1731] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:05:48.101144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:48.102101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:48.108095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.108543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.109300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.109735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.138972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aecb5c3f-9f78-447e-93b5-5b1a552f7520 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041845 May 23 16:05:48.139613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 920/1732] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:48.181556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:48.182474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:48.189414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.189866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.190620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.191050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.218587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1bf386c-e07e-4bbc-8bdb-2b0d9107ce65 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040937 May 23 16:05:48.219889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 813/1733] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:48.257717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:48.258268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:48.264498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.264947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.265698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:48.266132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:48.324980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:48.332484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aa0fd633-ee11-4180-9bf7-733abc891caa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078674 May 23 16:05:48.333033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 921/1734] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:49.150489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:49.151473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:49.157468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.157921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.158682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.159137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.187313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e4810f4d-df4a-4f6c-a64e-bdf9e561852b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040777 May 23 16:05:49.187893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 814/1735] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:05:49.230907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:49.231985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:49.238831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.239315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.240409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.240911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.271437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bf86013-40c5-4cef-9411-25238d7a72e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044405 May 23 16:05:49.271992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 922/1736] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:49.348974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:49.349588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:49.355580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.356023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.356772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:49.357218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:49.419564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:49.427518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-90042339-f3bf-4ea2-a5d1-99ddc581e1e1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082385 May 23 16:05:49.428076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 815/1737] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:50.199029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:50.200017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:50.206515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.206970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.207759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.208199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.239507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0efc4487-133d-457a-9bb5-d3368559a372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044313 May 23 16:05:50.240118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 923/1738] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:50.283087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:50.284046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:50.291860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.292310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.293107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.293538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.323661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21c16f73-49e9-41cc-a6e7-3f4418d22c31 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045102 May 23 16:05:50.329417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 816/1739] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:50.444671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:50.445283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:50.451176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.451623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.452367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:50.452795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:50.512215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:50.519838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-537a4f2f-4ad2-4768-a26b-6b82ac9157eb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079081 May 23 16:05:50.520390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 924/1740] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:50 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:51.251550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:51.252504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:51.254413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=92 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:51.259148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.259604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.260362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.260796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.289959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f4e7b840-0289-4c6d-837a-89e40875ab69 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042922 May 23 16:05:51.290579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 817/1741] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:51.335348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:51.336273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:51.342794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.343284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.344037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.344521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.372813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bf7ed6cc-9422-4fff-9d68-a0228d3a1385 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041507 May 23 16:05:51.373365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 925/1742] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:51.537576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:51.538120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:51.544344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.544793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.545552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:51.545987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:51.606294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:51.614382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef96fc12-1823-4ab3-af89-e6e3cb29af83 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080804 May 23 16:05:51.614944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 818/1743] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:52.301622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:52.302537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:52.308517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.308985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.309735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.310159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.338457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7dae377-1fee-42ab-b8ef-9d7c68626bf2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041359 May 23 16:05:52.339045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 926/1744] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:52.385086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:52.386005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:52.393638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.394082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.394844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.395303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.424096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-76c73136-1e82-4ec4-813d-75f23c1a49fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042903 May 23 16:05:52.424648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 819/1745] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:52.632452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:52.632996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:52.639222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.639680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.640445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:52.640889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:52.701767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:52.709194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3643afdc-693f-4a67-a322-dd60ae006cd6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081219 May 23 16:05:52.709772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 927/1746] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:53.349827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:53.350753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:53.356776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.357227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.357986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.358427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.387064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5bfec12b-d248-480c-989c-83bf6b0afcbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041182 May 23 16:05:53.387686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 820/1747] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:53.435456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:53.436401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:53.443013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.443501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.444262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.444704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.472836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ba356b7c-c9e6-4690-becc-8edd2a06ba54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041391 May 23 16:05:53.473377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 928/1748] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:53.726240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:53.726786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:53.733191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.733643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.734399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:53.734833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:53.798842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:53.806276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-339fefb8-587a-4a03-8d05-53c4d7695060 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083888 May 23 16:05:53.806835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 821/1749] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:54.399284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:54.400245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:54.406824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.407339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.408138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.408580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.436455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a4c1af2a-a6c8-45a4-aea4-811e1179aaa8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041414 May 23 16:05:54.437079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 929/1750] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:54.484227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:54.485157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:54.491211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.491688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.492445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.492882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.521719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9eeae40b-3437-416d-9a65-bc821f30ea69 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041407 May 23 16:05:54.522256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 822/1751] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:54.822961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:54.823608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:54.829558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.830007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.830762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:54.831227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:54.890825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:54.898298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c95a654c-054e-4b10-8fe5-9c7fca34f2fa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079185 May 23 16:05:54.898866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 930/1752] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:55.448534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:55.449464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:55.455955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.459007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.459007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.459007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.486054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-40c49e9a-0559-4372-acd9-55b02b0a8018 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041635 May 23 16:05:55.486646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 823/1753] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:55.533736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:55.534668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:55.541045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.541500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.542257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.542696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.566302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:55.570686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-383c817f-2186-4870-b3d6-2570f1e1e30f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041490 May 23 16:05:55.571257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 931/1754] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:55.914832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:55.915478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:55.921438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.921889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.922655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:55.923136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:55.988173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:55.995601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4195197d-29af-40d7-957d-3ad0875bd128 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084568 May 23 16:05:55.996161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 824/1755] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:56.497876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:56.498733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:56.500553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:56.505623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:56.506070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:56.506834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:56.507304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:56.536465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-99e6c949-fa55-4410-be0b-a04f01e3160a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042927 May 23 16:05:56.537078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 932/1756] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:56.581745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:56.582671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:56.589127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:56.589575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:56.590325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:56.590760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:56.619955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a2a4306-4ab9-4226-af7c-b13397c5129d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042132 May 23 16:05:56.620522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 825/1757] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:57.012676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:57.013572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:57.020433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.020877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.021625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.022059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.125141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:57.132594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-81e85838-0c8d-43cf-b947-b1f50961b156 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.123777 May 23 16:05:57.133152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 933/1758] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:57.548588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:57.549507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:57.555518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.555965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.556727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.557160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.585171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-730e288d-ab1b-412c-977f-81bd079a3754 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041063 May 23 16:05:57.585768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 826/1759] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:57.631879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:57.632815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:57.639026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.639512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.640278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:57.640723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:57.668546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f695b12-e417-4e73-a2ba-57b2fe6ccf89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040626 May 23 16:05:57.669085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 934/1760] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:58.150135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:58.150775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:58.157762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.158204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.158953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.159421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.265194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:58.272812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ef5d049-e9d9-440d-9ebe-1efb78448f6e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.126531 May 23 16:05:58.273368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 827/1761] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:58.596828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:58.597671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:58.604168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.604618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.605372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.605804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.634600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b99edfc8-5993-418a-9371-0c8bb751a72c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041807 May 23 16:05:58.635218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 935/1762] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:58.679982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:58.680922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:58.686837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.687339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.688106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:58.688664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:58.717413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-df1a2bc4-5951-4e1e-8131-df8a91a6c148 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041349 May 23 16:05:58.717954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 828/1763] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:59.289542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:59.290148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:59.296154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.296594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.297338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.297768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.356774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:05:59.364252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f9d8dec3-e411-4e96-a9f4-1fcaf70e2275 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078606 May 23 16:05:59.364886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 936/1764] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:05:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:05:59.651440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:59.652322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:59.660619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.661075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.661838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.662274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.670325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:59.697285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55fe2887-58e2-4367-b6f5-9d12319affc6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054179 May 23 16:05:59.697911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 829/1765] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:05:59.729340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:05:59.733357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:05:59.737187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.737635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.738388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:05:59.738832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:05:59.768301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2136cc8-c72d-44f6-b888-1ce1f4ebb23f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042843 May 23 16:05:59.768851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 937/1766] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:05:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:00.381507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:00.382060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:00.388956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.389407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.390173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.390617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.449846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:00.458031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-12ea0677-714a-42ea-941f-31f3917c4ed4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080516 May 23 16:06:00.458599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 830/1767] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:00.709367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:00.710285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:00.716232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.716677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.717423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.717849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.746526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-39da4e91-a06c-4afd-874c-5dc13faa7a99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041212 May 23 16:06:00.747186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 938/1768] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:00.779638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:00.780574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:00.787674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.788130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.788894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:00.789335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:00.817468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2fd2a0e1-ec91-4ca9-9eec-3cde96206d49 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041734 May 23 16:06:00.818017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 831/1769] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:01.476384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:01.476996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:01.483073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.483545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.484336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.484776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.553112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:01.560773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6fcc5985-28ab-456c-806a-0bd557af105b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088926 May 23 16:06:01.561326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 939/1770] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:01.758364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:01.759234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:01.765456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.765900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.766677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.767122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.795851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6071caa2-d747-4561-93b1-7971f177fa08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041514 May 23 16:06:01.796471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 832/1771] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:01.828689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:01.829617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:01.835882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.836325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.837073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:01.837500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:01.866104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-43cd7676-e12e-4f01-8095-6dacb72da90a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041265 May 23 16:06:01.866633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 940/1772] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:02.578250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:02.578799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:02.584845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.585296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.586061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.586498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.649802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:02.657745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-26aaf109-b3aa-4c84-8076-9ef2e5a8104e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083064 May 23 16:06:02.658496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 833/1773] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:02.807745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:02.808671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:02.814897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.815384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.816153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.816591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.845726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff663423-64e3-481e-b29d-23e2cb9b5807 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041257 May 23 16:06:02.846308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 941/1774] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:02.877260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:02.878194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:02.884631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.885085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.885851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:02.886290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:02.915557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-526a1c97-1522-4380-9f36-3b7cf494d056 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042099 May 23 16:06:02.916099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 834/1775] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:03.674877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:03.675523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:03.681589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.682031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.682780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.683269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.746788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:03.754986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9990730-6d58-4162-a367-358e54c38a14 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084466 May 23 16:06:03.755910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 942/1776] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:03.857097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:03.858036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:03.864631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.865085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.865852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.866291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.894911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c21e4071-0942-4f10-b8e3-8ec2e48f0893 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041768 May 23 16:06:03.895535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 835/1777] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:03.927455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:03.928317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:03.934660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.935130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.935902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:03.936340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:03.969311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8d754dd-f600-4368-9a8d-55fc55086ebc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045952 May 23 16:06:03.970904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 943/1778] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:04.774320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:04.774932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:04.781262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.781709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:04.782982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.783450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:04.843571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:04.851658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e25e044-11f9-4625-9058-b6d4c6655fed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081174 May 23 16:06:04.852227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 836/1779] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:04.905819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:04.906856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:04.913333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.916698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:04.916698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.916698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:04.946475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1846a203-3a19-4a74-adef-33a051f33aa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044529 May 23 16:06:04.947087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 944/1780] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:04.981887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:04.982819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:04.990059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.990507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:04.991301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:04.991738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:05.020256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08a3f4c6-001f-4b2c-9ddb-762780b0411d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042440 May 23 16:06:05.020814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 837/1781] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:05.869338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:05.869947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:05.875971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:05.876411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:05.877158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:05.877587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:05.938399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:05.945744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34d99564-ab37-4378-805b-32102cee73ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080417 May 23 16:06:05.946298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 945/1782] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:05.958268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:05.959333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:05.965325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:05.965770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:05.966513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:05.966943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:06.002787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-20f14b86-1288-48d6-8add-c50f4b3194f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048374 May 23 16:06:06.003297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 838/1783] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:06.037278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:06.038199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:06.044205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:06.044648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:06.045390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:06.045821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:06.073889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-15d844aa-40a6-4f9e-aa00-259f8c966930 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040554 May 23 16:06:06.074465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 946/1784] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:06.963531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:06.964144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:06.970177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:06.970627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:06.971418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:06.971854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:07.014200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:07.015283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:07.022251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:07.022695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:07.023489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:07.023921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:07.036071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:07.043559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ba54f230-15e8-4c1e-b73d-98805ca04267 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083842 May 23 16:06:07.044121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 839/1785] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:07.052416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0859e44b-9e27-4209-a056-e3409513c67d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042465 May 23 16:06:07.052969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 947/1786] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:07.085586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:07.086507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:07.092524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:07.092974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:07.093732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:07.094170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:07.121695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d84aabb5-ed4a-4947-9e44-11d48eb23dbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039924 May 23 16:06:07.122236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 840/1787] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:08.060549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:08.061275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:08.063537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:08.064467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:08.067825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.068282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.069046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.069486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.070955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.071446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.072211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.072647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.087820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:08.101930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a83d0553-9706-41b2-b917-a5acc75e0eaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042204 May 23 16:06:08.106476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 841/1788] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:08.132020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:08.132323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:08.133350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:08.139645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.139802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-622e2bf9-a8d7-4404-bcc3-346335b5b760 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083170 May 23 16:06:08.140090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.140304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 948/1789] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:08.140847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:08.141790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:08.170207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a655100-f805-4489-bd44-339f803d2e94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041933 May 23 16:06:08.170943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 842/1790] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:09.116647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:09.117568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:09.124139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.124585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.125339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.125842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.154282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b70b38f3-d54e-456e-ab82-dc05e36669fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041636 May 23 16:06:09.154827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 949/1791] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:09.156048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:09.156668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:09.163745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.164198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.164958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.165392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.181545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:09.182458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:09.188827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.189276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.190034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:09.190461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:09.220302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a732a4a3-db38-44b4-a4d6-b667a735bc5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041933 May 23 16:06:09.220862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 950/1792] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:09.227708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:09.235370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d5336a7a-67a3-432c-be08-566f3661ee7c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082982 May 23 16:06:09.235925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 843/1793] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:10.165755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:10.166705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:10.173167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.173615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.174363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.174792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.202890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ff4a461-52b6-4f56-a35e-400b68e81a98 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041048 May 23 16:06:10.203465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 951/1794] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:10.231602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:10.232544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:10.239557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.240011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.240770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.241204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.252101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:10.252719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:10.258666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.259056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.259851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:10.260291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:10.269808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-43b54928-2b5b-4c01-b37d-5b6fb5fa6187 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042101 May 23 16:06:10.270373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 844/1795] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:10.320874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:10.328373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ce02f76-dfdb-48e9-b92a-1fc7b32d739d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080137 May 23 16:06:10.329433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 952/1796] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:11.214833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:11.215779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:11.222548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.223001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.223795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.224233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.249258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:11.253661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a07e3b5-cbb1-4f9b-b19c-dcbac773f29d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042780 May 23 16:06:11.254217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 845/1797] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:11.281241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:11.282827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:11.289230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.289680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.290437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.290872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.318613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8cdfc2cc-4de2-4ad0-b294-7768df709dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041303 May 23 16:06:11.319184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 953/1798] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:11.345910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:11.346533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:11.352608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.353056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.353822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:11.354259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:11.416692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:11.424847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c03d234-32ee-40a5-b45d-0f7e86127d0f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083914 May 23 16:06:11.425411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 846/1799] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:12.264711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:12.265590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:12.271710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.272167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.272926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.273366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.302494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4bcb82b-c8a3-4cd2-964e-1a02b37a15ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042108 May 23 16:06:12.303134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 954/1800] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:12.329986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:12.330911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:12.337717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.338176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.338952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.339432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.368360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06cce330-16d8-401d-bfd6-29bfc017a08d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042363 May 23 16:06:12.368930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 847/1801] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:12.441340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:12.442044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:12.448268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.448784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.449550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:12.450051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:12.510918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:12.519329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31b10f7d-cb5a-48b1-b7c5-05712958b9a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082177 May 23 16:06:12.519832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 955/1802] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:13.314941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:13.315883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:13.322146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.328033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.328033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.328033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.355582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e3be476-c202-455d-a06c-264f97f58371 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045535 May 23 16:06:13.356419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 848/1803] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:13.380066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:13.381006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:13.387438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.387888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.388920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.389358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.418412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-80a97c2f-e408-42e9-b27f-4cb6dcc3a328 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042622 May 23 16:06:13.418964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 956/1804] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:13.535686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:13.536303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:13.543586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.544031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.544785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:13.545214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:13.607193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:13.615192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae528900-d06f-4190-981f-2f127aa4b2d1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083481 May 23 16:06:13.615755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 849/1805] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:14.367305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:14.368167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:14.374174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.374632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.375421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.375864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.404553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d86f8acf-0ff7-48cb-ad7d-ea9528f3b0a0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041396 May 23 16:06:14.405235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 957/1806] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:14.429723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:14.430653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:14.437085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.437535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.438910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.439375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.466898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ab58962a-3390-43c6-b6e5-ecbfaa2c8d17 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041121 May 23 16:06:14.467472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 850/1807] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:14.631984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:14.632523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:14.638722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.639204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.639964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:14.640400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:14.699509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:14.706884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a68da3ff-07d6-4769-9fa6-97c5599dc638 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078823 May 23 16:06:14.707481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 958/1808] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:15.416594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:15.417644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:15.424130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.424584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.425349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.425785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.454267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c48072c4-c9fa-49a2-a158-d0ddec0fd789 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041586 May 23 16:06:15.454910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 851/1809] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:15.477832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:15.478768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:15.485844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.486295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.487052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.487553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.516625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7205f71b-f7e4-4a2f-9696-e3f2b5665891 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042642 May 23 16:06:15.517381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 959/1810] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:15.724083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:15.724704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:15.730652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.731136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.731913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:15.732353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:15.794310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:15.801853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f59f72ba-1431-40a3-85c8-4e624cc69607 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082228 May 23 16:06:15.802419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 852/1811] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:16.465723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:16.466639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:16.472692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.473138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.473894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.474356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.504035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0add66f9-a619-4eaa-841e-86b1f9841afc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042188 May 23 16:06:16.504728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 960/1812] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:16.528133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:16.529059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:16.535444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.535893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.536643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.537071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.565936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-43259bd9-5013-4bee-b49b-e4e7d80345cf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041747 May 23 16:06:16.566474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 853/1813] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:16.818239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:16.818785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:16.824805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.825258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.826019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:16.826455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:16.886230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:16.894578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b39cad4-353f-466b-92d7-b2ce66b269b7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080238 May 23 16:06:16.895153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 961/1814] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:17.515929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:17.516809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:17.522803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.523291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.524053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.524491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.554310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb5fe3bf-9daa-488f-bce4-9873620995ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042730 May 23 16:06:17.554923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 854/1815] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:17.576643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:17.577575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:17.583893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.584334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.585080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.585509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.614338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6313f5cf-a444-4da3-bd0f-72a0b7db06a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041582 May 23 16:06:17.614881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 962/1816] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:17.911789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:17.912355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:17.918320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.918772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.919558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:17.919996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:17.979513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:17.986953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75f37545-533f-47d8-bc52-554f2bc0fc37 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079781 May 23 16:06:17.987546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 855/1817] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:18.566056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:18.566914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:18.572939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:18.573393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:18.574155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:18.574600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:18.603599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3a87cf09-8938-49b4-869f-84615baadaab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041523 May 23 16:06:18.604211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 963/1818] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:18.625694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:18.626632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:18.633071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:18.633523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:18.634287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:18.634729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:18.663919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-67f9b501-5792-4376-aaf4-c4d5520417fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042142 May 23 16:06:18.664460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 856/1819] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:19.002853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:19.003643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:19.009628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.010075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.010827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.011308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.071775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:19.079248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9d12b4e7-8d4d-43f6-ad91-3ad4c6a0ccd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080206 May 23 16:06:19.079810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 964/1820] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:19.615759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:19.616611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:19.623298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.623762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.624593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.625062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.655747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b8e8c5e-64e5-48c1-899a-649b51689ac9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043991 May 23 16:06:19.657862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 857/1821] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:19.675929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:19.676866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:19.683453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.683904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.684669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:19.685107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:19.689438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:19.714384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f3ec1251-e5a0-47cb-9706-f1c174911fc1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042876 May 23 16:06:19.714949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 965/1822] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:20.096436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:20.097046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:20.103011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.103490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.104414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.104853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.169044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:20.180397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e584847-41cd-40e0-8fbe-fb90ee9b21cf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087767 May 23 16:06:20.181006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 858/1823] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:20.666469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:20.667427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:20.674553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.675014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.675802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.676242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.704435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dedc3e39-bb20-4d70-8b81-bc8f26b5583f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041906 May 23 16:06:20.704985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 966/1824] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:20.726462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:20.727449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:20.733873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.734324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.735732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:20.736167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:20.765266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68e949d5-3d09-4c17-a1ab-e52789de619f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042874 May 23 16:06:20.765818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 859/1825] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:21.199779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:21.200430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:21.207481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.208155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.209292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.209944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.280482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:21.288553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-accc3179-2fb1-42cc-a38a-d38e708893fc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.092652 May 23 16:06:21.289115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 967/1826] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:21.716004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:21.716924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:21.718878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=86 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:21.724175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.724619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.725366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.725792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.754507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17e10e98-b379-45b2-9df6-bef6d120e2fd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042427 May 23 16:06:21.755082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 860/1827] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:21.776478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:21.777399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:21.784377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.784818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.785574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:21.786004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:21.813709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-447bc3f6-9d60-490f-95bd-00d75c606887 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041070 May 23 16:06:21.814261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 968/1828] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:22.306002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:22.306541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:22.312649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.313103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.313866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.314305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.373571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:22.380896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-858c9a58-b535-447f-b696-66fd0849d482 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078917 May 23 16:06:22.381463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 861/1829] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:22.766120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:22.767044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:22.773145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.773592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.774337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.774768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.804475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b9e27bb7-5a7c-4f36-96c6-10eaded020f5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042509 May 23 16:06:22.805063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 969/1830] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:22.825073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:22.825993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:22.832326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.832772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.833521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:22.833951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:22.850400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:22.862847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e20e3876-ca61-4294-b8ab-b8bbccec6f5f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041642 May 23 16:06:22.863419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 862/1831] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:23.398121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:23.398734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:23.404762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.405204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.405945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.406371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.470411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:23.477801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e161e41-efc6-4078-bac3-caba06ea2d36 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083499 May 23 16:06:23.478359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 970/1832] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:23.816492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:23.817410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:23.823838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.824283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.825030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.825457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.854090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1eaf09b8-301c-4fd5-bf01-f83d7bf607f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041539 May 23 16:06:23.854620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 863/1833] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:23.873727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:23.874666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:23.881078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.881532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.882295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:23.882731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:23.911969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-633c60a8-6309-4ccb-8761-bd09c033bec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042204 May 23 16:06:23.912517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 971/1834] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:24.494700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:24.495282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:24.501322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.501777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.502541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.502984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.565544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:24.573001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-866c81e1-4e92-460b-8d3f-c455ab1fbc99 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082287 May 23 16:06:24.573564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 864/1835] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:24.864675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:24.865538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:24.871561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.872012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.872771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.873203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.902600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-86c8271c-2fea-43be-ada0-dc95c933ae06 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041920 May 23 16:06:24.903244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 972/1836] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:24.923387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:24.924325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:24.932151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.932607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.933372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:24.933810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:24.968069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-94f751aa-24c6-4034-b6e7-25e722cb5376 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048643 May 23 16:06:24.968631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 865/1837] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:25.590652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:25.591207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:25.597196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.597647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:25.598398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.598831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:25.659946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:25.667379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca0f7d2d-ffee-465f-a0c6-4ad59154f6af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080650 May 23 16:06:25.667939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 973/1838] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:25.914342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:25.915234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:25.921580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.922210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:25.922971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.924091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:25.952934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-009c25d5-dd8a-4c32-935e-cd7a5cda2846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042576 May 23 16:06:25.953470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 866/1839] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:25.979439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:25.980370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:25.987350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.987802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:25.988549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:25.988978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:26.017016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0f5a5454-4e86-4f75-9029-e71712888edb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041427 May 23 16:06:26.018100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 974/1840] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:26.684830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:26.685359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:26.691843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:26.692286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:26.693043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:26.693473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:26.752739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:26.759986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8a6cbe4-c792-4f70-8bca-6729c3fbfb5b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079190 May 23 16:06:26.760541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 867/1841] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:26.964370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:26.965212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:26.967069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:26.972323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:26.972770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:26.973530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:26.973965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:27.002480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-416a452b-022c-46a7-906a-7adb3a8a3ac5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042073 May 23 16:06:27.003275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 975/1842] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:27.028740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:27.029694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:27.036693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:27.037141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:27.037912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:27.038370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:27.066098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f820d98b-824d-4db3-92da-cdb9f6b7d50a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041234 May 23 16:06:27.066637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 868/1843] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:27.777357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:27.777974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:27.784558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:27.785007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:27.785761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:27.786200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:27.844261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:27.852268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d087bbc0-e4f6-470a-800d-2d906fd79255 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078798 May 23 16:06:27.852847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 976/1844] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:28.014560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:28.015065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:28.021795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.022250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.023022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.023495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.059024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-01d1afb3-34e1-4159-8721-98a14e6eea3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049005 May 23 16:06:28.059710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 869/1845] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:28.077372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:28.078292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:28.085286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.085728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.086479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.086910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.114443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4308049d-6b1e-4fef-9ae1-8b3f06e33b50 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040920 May 23 16:06:28.115010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 977/1846] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:28.868873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:28.869482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:28.875422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.875869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.877128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:28.877546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:28.939751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:28.947133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8ee555f0-4c58-4840-a960-540bdc80ee73 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082051 May 23 16:06:28.947711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 870/1847] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:29.071126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:29.072044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:29.078023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.078579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:29.079249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.079745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:29.107340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2557e2e8-9a41-4544-b02e-4c18d864a8c6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040120 May 23 16:06:29.107876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 978/1848] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:29.125734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:29.127208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:29.133593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.134046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:29.134807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.135285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:29.163314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b5767bf5-0f1b-4671-8159-ad3db3a401d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041468 May 23 16:06:29.163859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 871/1849] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:29.964375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:29.964983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:29.970852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.971907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:29.972655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:29.973085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:30.032553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:30.040693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24c2dbf0-139f-4932-8126-e888f377cb3a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080150 May 23 16:06:30.041256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 979/1850] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:30.118864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:30.119835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:30.125874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:30.126445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:30.127084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:30.128054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:30.156561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-174a1d38-09bd-49c5-bc0c-cb6a1c95b9a4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041611 May 23 16:06:30.157115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 872/1851] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:30.173759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:30.174694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:30.181206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:30.181653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:30.182402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:30.182831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:30.211780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ebffafcf-d3d2-4428-87a2-25abf850742e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041919 May 23 16:06:30.212312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 980/1852] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:31.057507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:31.058120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:31.064239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.065205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.065961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.066397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.125704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:31.134103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f6ba1356-413d-48a4-a487-d933d72619fb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080438 May 23 16:06:31.134659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 873/1853] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:31.167925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:31.168847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:31.174809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.175399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.176027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.176458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.200493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:31.204825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-57c33907-0e1e-4e93-ab71-ab588255bcc2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040846 May 23 16:06:31.205484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 981/1854] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:31.223011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:31.223977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:31.230007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.230461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.231251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:31.231691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:31.260667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abe7145b-78d2-4cf3-83cf-71dd11e55f56 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042061 May 23 16:06:31.261215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 874/1855] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:32.151889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:32.152458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:32.159496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.159976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.160737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.161174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.222545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:32.223001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:32.223638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:32.230097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.230251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3bbf7267-9b9e-4e74-8fca-329d0c110a7e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082193 May 23 16:06:32.230551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.230674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 982/1856] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:32.231348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.231996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.260489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07168009-a1a9-4ed3-8aeb-f0da32d9c1f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042423 May 23 16:06:32.261036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 875/1857] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:32.272824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:32.273795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:32.279879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.280324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.281076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:32.281506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:32.309790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96fecd20-0044-4b9c-b56e-513308586920 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040898 May 23 16:06:32.310327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 983/1858] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:33.247681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:33.248306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:33.259255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.259704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.260474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.260903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.272304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:33.273233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:33.279716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.280173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.280934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.281409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.311844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-abda7e5e-ffad-4b02-8a1f-10e2e64a61ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043357 May 23 16:06:33.312401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 984/1859] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:33.319702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:33.320613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:33.324577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:33.326344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.326792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.327568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:33.328001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:33.332049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-717924fa-3ba8-4cfd-81c8-f7f1be09e54e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088225 May 23 16:06:33.332617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 876/1860] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:33 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:33.356377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-60dadd57-c0ff-4d04-82ac-febb1b7aa4f9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040217 May 23 16:06:33.356912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 985/1861] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:34.323523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:34.324399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:34.331406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.331858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.332619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.333057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.349120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:34.349859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:34.356211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.356660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.357421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.357855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.362715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68ca1a03-6c46-4e6e-b349-fe83d5c27811 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043266 May 23 16:06:34.363311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 877/1862] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:34.367795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:34.368711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:34.374420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.374873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.375679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:34.376118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:34.405586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f5a86794-f895-42a5-8ba4-23586714ea05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041247 May 23 16:06:34.406344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 878/1863] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:34.418757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:34.426306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-80295a09-7925-4a2c-9453-1ef454c0aff7 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081542 May 23 16:06:34.426857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 986/1864] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:35.373875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:35.374797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:35.381883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.382329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.383078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.383543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.387142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:35.412048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abfbe804-82f9-41c1-bb87-0b8ea69a74fe tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042149 May 23 16:06:35.412592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 879/1865] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:35.417837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:35.418769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:35.425919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.426364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.427138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.427579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.442651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:35.443288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:35.449513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.449959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.450714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:35.451174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:35.456042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1f1c58fa-89bb-4ce6-b27e-1b1e04b2e09b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042051 May 23 16:06:35.456600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 987/1866] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:35.510985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:35.518783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08325877-4adc-4184-9c87-b59f33189a6d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079978 May 23 16:06:35.519371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 880/1867] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:36.423870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:36.425305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:36.431412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.431758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.432518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.432948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.462229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-def15a2a-4c25-4653-a6c1-57925c318818 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042605 May 23 16:06:36.462783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 988/1868] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:36.472134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:36.473065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:36.479620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.480073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.480837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.481272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.511781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8a3fecb1-c81e-4269-961c-c182eac06109 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045683 May 23 16:06:36.512334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 881/1869] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:36.535346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:36.535982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:36.542163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.542602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.543374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:36.543802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:36.605397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:36.614702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-695517d7-ca7e-402c-97b0-c83f83fc7d85 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083784 May 23 16:06:36.615299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 989/1870] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:37.474439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:37.475391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:37.481792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.482236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.482979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.483436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.510571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a1bd8854-55fd-414a-a155-7d3a08ad3f6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040770 May 23 16:06:37.511073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 882/1871] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:37.523240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:37.524213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:37.530509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.530953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.531741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.532175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.560857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2367b230-153b-4000-90f4-86a7332c1ed8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041560 May 23 16:06:37.561399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 990/1872] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:37.631410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:37.632020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:37.637941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.638388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.639431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:37.639864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:37.704836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:37.712351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6391588c-cacc-4f8d-865d-c230adbd2950 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085314 May 23 16:06:37.712906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 883/1873] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:38.522952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:38.523910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:38.530123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.530571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.531364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.531803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.559406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0b5da98-638c-4754-a13c-0d9fc89fa3f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040391 May 23 16:06:38.559949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 991/1874] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:38.572039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:38.572966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:38.579325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.579770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.581103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.581534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.608957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc7717b4-2d0e-4bcb-87ad-8941a685030a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040780 May 23 16:06:38.609492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 884/1875] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:38.729575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:38.730186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:38.736161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.737457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.738619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:38.739296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:38.799698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:38.806990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9fa86d81-0fbb-4ae0-aad9-926208c16d7d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081223 May 23 16:06:38.807589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 992/1876] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:39.571000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:39.571864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:39.578262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.578833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.579470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.579905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.608586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-023cc413-35ba-4310-add0-47da0da9b22f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041682 May 23 16:06:39.609119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 885/1877] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:39.620174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:39.621139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:39.627608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.628059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.628886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.629333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.659025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cddbcc93-9845-45c6-9c59-61380f5f8779 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042151 May 23 16:06:39.659692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 993/1878] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:39.824356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:39.824897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:39.831371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.831820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.832580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:39.833017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:39.892487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:39.900557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d86d5e93-ac18-4933-8841-3fb6eafff406 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080079 May 23 16:06:39.901118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 886/1879] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:39 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:40.619998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:40.620842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:40.627000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.628014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.628765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.629198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.657375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-94bce94a-03e5-491d-b801-f03415e45c0a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041409 May 23 16:06:40.657906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 994/1880] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:40.670019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:40.670986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:40.678130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.678586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.679387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.679835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.708367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5bfe277-f4a2-4fca-a961-487d6fc4e9cb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042390 May 23 16:06:40.708921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 887/1881] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:40.917524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:40.918063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:40.924666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.925118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.925874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:40.926309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:40.985509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:40.993493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-adac9691-cbab-4f79-b4f8-4885792c8fd5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079890 May 23 16:06:40.994051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 995/1882] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:41.669367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:41.670302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:41.677105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:41.677561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:41.678327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:41.678769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:41.706504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0b13ab5-e8c8-4293-a84a-deaf8b53eb97 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041099 May 23 16:06:41.719197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 888/1883] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:41.719656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:41.720594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:41.727854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:41.728308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:41.729070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:41.729510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:41.758236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e25a59c-f78f-40ea-859b-ea11e3a96502 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042372 May 23 16:06:41.758714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 996/1884] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:42.010335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:42.010905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:42.017406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.017859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.018628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.019123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.082909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:42.091507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3e5e7787-0172-4739-aeba-6bddec73ebac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084706 May 23 16:06:42.092549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 889/1885] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:42.718090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:42.719022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:42.726172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.726778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.727413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.727857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.755690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a6f21b53-0af3-44a8-854c-6ec16486a9ec tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041637 May 23 16:06:42.756235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 997/1886] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:42.769775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:42.771283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:42.777765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.778217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.779006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:42.779484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:42.808385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7a0264a-99b3-4977-9280-76f671ffa7f5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042489 May 23 16:06:42.808948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 890/1887] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:43.109881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:43.110509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:43.116518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.116973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.117740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.118181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.136132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:43.178813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:43.186688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a9faf41b-63cc-468c-886a-48dbc97b6a3c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080665 May 23 16:06:43.187313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 998/1888] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:43.767392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:43.768283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:43.774288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.774875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.775525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.775965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.805255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-294e8340-56c8-4050-a56a-6d54d64a6895 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042109 May 23 16:06:43.805796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 891/1889] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:43.819828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:43.820783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:43.827441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.827900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.828697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:43.829136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:43.857515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a6c6cab8-49b4-4cf6-80d3-37799688d853 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041757 May 23 16:06:43.858058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 999/1890] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:43.941485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:06:43.941652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:06:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:06:44.204318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:44.204927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:44.211360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.211805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.212553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.212981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.272447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:44.279899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f4d97612-6692-4b27-806e-589c4f67b271 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079346 May 23 16:06:44.280392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 892/1891] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:44.616830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:06:44.617004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:06:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:06:44.816883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:44.817806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:44.824861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.825412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.826051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.826489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.855755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-113e711b-3156-4f88-ab35-db0a24904226 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042878 May 23 16:06:44.856298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1000/1892] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:44.869603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:44.870471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:44.876925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.877366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.878115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:44.878544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:44.907865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e49ecd9-56ae-4acb-87e2-b3731e928b84 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042871 May 23 16:06:44.908405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 893/1893] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:45.297085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:45.297697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:45.304600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.305045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.305800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.306233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.364652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:45.372556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9d9e5dfe-ce4a-4096-857a-368717c95bb6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079295 May 23 16:06:45.373143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1001/1894] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:45.867711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:45.868574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:45.875002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.875605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.876218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.876649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.907456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d08ace79-836b-4e19-a98d-a32d9b8ffa5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043707 May 23 16:06:45.912438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 894/1895] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:45.919158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:45.920144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:45.926468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.926913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.927712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:45.928147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:45.956694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1d66a50c-caee-46a8-afe3-4bfe1326b40f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041502 May 23 16:06:45.957327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1002/1896] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:46.389929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:46.390459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:46.396836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.397287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:46.398036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.398569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:46.461462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:46.464825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:46.469081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15ffd712-d88b-46e9-8db4-457d16ac8c57 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083297 May 23 16:06:46.469638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 895/1897] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:46.921416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:46.922344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:46.928831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.929287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:46.930079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.930518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:46.958628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-deafc1c9-3a3b-4178-8449-ea50e6d7adb3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041184 May 23 16:06:46.959189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1003/1898] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:46.968940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:46.969875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:46.976467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.976919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:46.978328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:46.978765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:47.006556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e19ef88c-60f7-4571-a003-5849cb6ec77d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041948 May 23 16:06:47.007112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 896/1899] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:47.485779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:47.486337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:47.492391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:47.492840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:47.493598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:47.494031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:47.553598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:47.560996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-37a4f182-1327-4977-87a1-19f4813506a6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079162 May 23 16:06:47.561551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1004/1900] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:47.970749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:47.971643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:47.978763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:47.979247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:47.980007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:47.980440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:48.008169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9d166ab-16d9-4340-8239-4658952bec67 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041421 May 23 16:06:48.008709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 897/1901] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:48.018002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:48.018919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:48.025853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:48.026299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:48.027048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:48.027504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:48.055459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-356c2b3a-7ef9-41b1-9692-db1fa4e1aa28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041646 May 23 16:06:48.056075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1005/1902] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:48.578363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:48.578893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:48.584924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:48.585374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:48.586126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:48.586560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:48.645681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:48.653022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-037830ce-40f1-4645-a48c-e8c7001a443d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078621 May 23 16:06:48.653581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 898/1903] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:49.020433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:49.021351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:49.028184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.028627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.029373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.029838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.060639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31514f6e-67af-4ef6-b238-10dbbfce9be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044227 May 23 16:06:49.061185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1006/1904] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:49.066958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:49.074591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:49.074591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.074972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.075773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.076208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.105389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1368a1a5-0620-4f69-a6c3-14315a4bde5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042415 May 23 16:06:49.105933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 899/1905] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:49.669817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:49.670347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:49.676442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.676894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.677649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:49.678085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:49.737434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:49.745216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-06cb49cd-5e3d-476e-a22d-46366da968e4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079315 May 23 16:06:49.746561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1007/1906] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:50.072717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:50.073651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:50.080104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.080555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.081313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.081748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.110283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10f8a485-f8f3-4853-a9e0-caf430ad0290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042149 May 23 16:06:50.111076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 900/1907] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:50.116000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:50.117207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:50.124704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.125254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.126530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.130229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.161019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-79b5e6eb-65fe-4dd0-9354-4039be727374 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048840 May 23 16:06:50.161576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1008/1908] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:50.761837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:50.762451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:50.768793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.769246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.770829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:50.770829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:50.833430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:50.841413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e943bb69-9a87-4737-a823-a6b754494ff4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083461 May 23 16:06:50.841970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 901/1909] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:50 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:51.122024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:51.122865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:51.129412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.129865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.130644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.131077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.161187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a3504f66-a51f-4434-805a-bf52b6291398 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043316 May 23 16:06:51.161749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1009/1910] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:51.172750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:51.173688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:51.180070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.180524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.181287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.181726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.210198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fd16ef37-f406-4c4b-96b1-f0084a8d85f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041718 May 23 16:06:51.210737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 902/1911] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:51.858675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:51.859331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:51.865331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.865787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.866546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:51.866981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:51.926480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:51.934475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fcdbbb56-8d7e-40a3-be6f-8e3e9a3b1bd4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079711 May 23 16:06:51.935037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1010/1912] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:52.172974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:52.173910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:52.180130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.180579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:52.181337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.181822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:52.209563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03534905-6d67-465a-aff5-739d8ecd78ce tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040732 May 23 16:06:52.210111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 903/1913] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:52.221720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:52.222649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:52.229956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.230409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:52.231209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.231651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:52.260209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e6de6da2-d95d-4249-80bf-d350a604fc01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042374 May 23 16:06:52.260761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1011/1914] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:52.951736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:52.952346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:52.954267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=88 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:52.959054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.959551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:52.960326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:52.960763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:53.021397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:53.029222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8fd7be1a-41e3-4ea7-ac92-bdd5d65332c0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081308 May 23 16:06:53.029811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 904/1915] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:53.221592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:53.222484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:53.229094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:53.229541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:53.230306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:53.230737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:53.260105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-946dc831-310d-4309-be84-d5c9c4ccf361 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042526 May 23 16:06:53.260659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1012/1916] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:53.271045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:53.272002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:53.278525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:53.278982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:53.279775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:53.280213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:53.309354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6222505-c66c-420c-8414-0f4f7d25f80e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042220 May 23 16:06:53.309892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 905/1917] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:54.045657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:54.046268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:54.052509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.052977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.053722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.054171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.113338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:54.122529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-abf8dfe7-0229-4fac-92e2-e6abb4c43715 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080727 May 23 16:06:54.123090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1013/1918] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:54.271800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:54.272722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:54.280191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.280578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.281332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.281761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.309901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-732a5a18-810b-4f12-9029-bf6772a169e2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042006 May 23 16:06:54.310481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 906/1919] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:54.320415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:54.321888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:54.328271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.328721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.329474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:54.329909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:54.358534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9169f80-e6d5-49ae-bf41-5fb59c779bfe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042006 May 23 16:06:54.359084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1014/1920] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:55.140268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:55.140875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:55.146978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.147457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.148213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.148714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.213723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:55.221361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cf015d70-6679-43c6-a3f7-4b0fd2af048b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085355 May 23 16:06:55.224798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 907/1921] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:55.322036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:55.322968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:55.329486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.329940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.330705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.331166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.334702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:55.359219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-18b9acb0-ae35-4e36-a01f-aed5cdf8908f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041733 May 23 16:06:55.359774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1015/1922] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:55.370387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:55.371371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:55.377320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.377769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.378553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:55.379020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:55.407705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53667a80-904c-452f-ad77-53a6be8da50c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041787 May 23 16:06:55.408249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 908/1923] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:56.238302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:56.238924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:56.245568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.246022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.246784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.247265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.307342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:56.314739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-249714fe-2a56-4a15-8574-9317de679a9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080322 May 23 16:06:56.315406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1016/1924] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:56.370564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:56.371517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:56.377921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.378373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.379162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.379602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.408795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5146fce-1da4-4cda-bd51-f77a9d6c9732 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042139 May 23 16:06:56.409339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 909/1925] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:56.419396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:56.420331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:56.426711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.427200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.427979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:56.428401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:56.456895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83176298-1c03-4057-88c2-440f37242055 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041851 May 23 16:06:56.457437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1017/1926] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:57.332189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:57.332820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:57.339414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.339874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.340629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.341067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.400350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:57.407844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a34e883-13d6-442e-b794-a05261c2a0d5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079533 May 23 16:06:57.408404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 910/1927] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:57.421123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:57.422050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:57.424342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:57.429005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.429454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.430261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.430732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.459278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b5a96af-13d8-4fba-a0b1-4b2694350a64 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042339 May 23 16:06:57.459825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1018/1928] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:57.468701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:57.469631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:57.475557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.476012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.476774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:57.477210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:57.505596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ca63bcaa-977b-4f5d-8c54-74bfc19beb0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040769 May 23 16:06:57.506065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 911/1929] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:58.424657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:58.425196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:58.432543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.432983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.433725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.434151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.470967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:58.471915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:58.478860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.479334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.480096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.480521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.494413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:58.497918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:58.502006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-22124256-5ee9-4343-bd1c-daab4025b082 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081242 May 23 16:06:58.502572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1019/1930] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:06:58.510211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-51056cc8-e888-4e92-991d-f06a7ec70950 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043042 May 23 16:06:58.517274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 912/1931] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:06:58.517472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:58.518512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:58.524568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.525019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.525806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:58.526247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:58.554442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-776c9f63-e1c4-44f7-9d06-4fda8a4d7a58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041229 May 23 16:06:58.554978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1020/1932] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:59.520594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:59.521147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:59.526306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:59.527258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:59.527717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.528162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.528923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.529352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.533777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.534229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.534987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.535454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.564689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2b778861-b005-4e65-9639-1743ddad0e8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042232 May 23 16:06:59.565256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1021/1933] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:06:59.569841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:06:59.570762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:06:59.577229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.577680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.578446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:06:59.578880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:06:59.592701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:06:59.600061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08b0cb07-e7fe-48b8-8020-4059a84fac7f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083789 May 23 16:06:59.600617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 913/1934] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:06:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:06:59.607039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8f9de8f0-5313-4d66-973e-f6765f59de72 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040777 May 23 16:06:59.607610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1022/1935] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:06:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:00.575257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:00.576185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:00.582625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.583072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.583847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.584278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.613232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b1ebad82-4f7d-415b-8469-f5114c08f4f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041937 May 23 16:07:00.613783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 914/1936] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:00.616667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:00.617313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:00.618190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:00.619087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:00.623967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.624420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.625093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.625223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.625552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.625673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.626355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:00.626795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:00.656714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84ed3629-8d13-44dc-83ae-77805fa5e9d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041937 May 23 16:07:00.657500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 915/1937] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:00.685919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:00.693513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9326e7c8-fbde-4c0c-81da-2cc14bfaef7a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080611 May 23 16:07:00.694155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1023/1938] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:01.624820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:01.625545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:01.631908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.634002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.634002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.634002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.667732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:01.668225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9447396a-3f3e-4588-85bc-27e79e5190ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047933 May 23 16:07:01.668659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:01.672934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 916/1939] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:01.676221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.676668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.677573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.677847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.707688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f47520aa-8a79-4738-b68d-d7fe68b425f6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043970 May 23 16:07:01.708256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1024/1940] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:01.716755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:01.717921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:01.724102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.724555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.725483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:01.725920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:01.787274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:01.796055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-14be0a14-dc41-4e66-a3a1-e009c4ae9a94 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082451 May 23 16:07:01.796614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 917/1941] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:02.680216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:02.681138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:02.688062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.688509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.696989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.696989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.718606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f85868b0-2250-4d3c-8353-a9ef19d39f38 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042276 May 23 16:07:02.719199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1025/1942] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:02.719375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:02.720133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:02.726477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.726924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.727722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.728149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.756363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-793c6dc2-d575-4647-865a-7cacc639b982 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041030 May 23 16:07:02.756900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 918/1943] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:02.813765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:02.814386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:02.820331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.820776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.821529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:02.821961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:02.880923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:02.888324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eeeb1f8d-177e-4c65-8f4b-90e8fca4ffdb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078970 May 23 16:07:02.889132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1026/1944] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:02 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:03.730299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:03.731219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:03.737965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.738410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.739191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.739650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.768534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:03.769457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:03.769741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9172894-d940-4e6e-b6ed-0cb1912e7e30 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043551 May 23 16:07:03.770311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 919/1945] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:03.775900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.776340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.777091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.777518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.806088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-be893102-8038-4227-87a7-914765c68c8d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042152 May 23 16:07:03.806621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1027/1946] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:03.905083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:03.905701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:03.912377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.912833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.913595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:03.914033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:03.975556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:03.983019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3afd3c39-de74-4fda-a58e-0b2b717564ed tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081759 May 23 16:07:03.983610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 920/1947] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:04.781259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:04.782693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:04.789286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:04.789743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:04.790502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:04.790969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:04.817583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:04.818509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:04.819699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9e8e962a-f3cd-4708-86c2-f1ba895de8b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042342 May 23 16:07:04.820260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1028/1948] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:04.824883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:04.825332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:04.826087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:04.826518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:04.854733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-494be8ad-66fa-40c9-9014-195bc65ac9c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041166 May 23 16:07:04.855298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 921/1949] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:04.998878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:04.999530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:05.005712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.006160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.006920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.007385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.068600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:05.076256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f2c4ea3-f618-4fd9-8709-6d6435211f8d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081194 May 23 16:07:05.076819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1029/1950] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:05.831264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:05.832198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:05.838615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.839068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.839852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.840287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.866366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:05.867341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:05.868610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75c16264-5d5b-4d4a-8b88-9c9ed3356edf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041301 May 23 16:07:05.869162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 922/1951] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:05.873626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.874081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.874850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:05.875322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:05.904260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f658ab98-4e1a-4c88-b3ce-322f197b6161 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041856 May 23 16:07:05.904798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1030/1952] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:06.093567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:06.094402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:06.100450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.100903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.101683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.102098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.171207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:06.179288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dde04f02-5863-4712-bb04-4850abeb13af tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.089564 May 23 16:07:06.183632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 923/1953] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:06.879682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:06.880607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:06.886979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.887457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.888203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.888631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.913755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:06.915750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:06.916676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:06.918180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8fbc4978-47ea-4915-ba25-d9d2ff4e7bfa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042521 May 23 16:07:06.918736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1031/1954] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:06.923553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.923997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.924755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:06.925185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:06.955352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-079c90c5-dcc2-4e3d-bff1-8186934096f3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042919 May 23 16:07:06.956089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 924/1955] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:07.196823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:07.197349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:07.203325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.203764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:07.205188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.205619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:07.265399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:07.273325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b80ad26-9dfc-470f-9d87-874cc2442d26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080848 May 23 16:07:07.273875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1032/1956] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:07.930086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:07.931007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:07.937463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.937852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:07.938599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.939028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:07.967071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:07.968170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:07.975652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.976101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:07.976879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:07.977311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:08.009812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9850b5a7-f30d-4192-9b52-4219a66061c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046635 May 23 16:07:08.010397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1033/1957] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:08.123453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f55c67ad-49b8-49f5-b066-68a677f74e4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.197228 May 23 16:07:08.123999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 925/1958] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:08.290432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:08.291079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:08.297115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:08.297598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:08.298357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:08.298794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:08.360343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:08.367731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-437d911c-5118-44d4-a7af-b4e2d553a451 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081129 May 23 16:07:08.368325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1034/1959] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:09.021930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:09.022876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:09.029186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.029631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.030377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.030808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.059051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ffc48b67-3358-4da6-9e5c-0573f39189ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041041 May 23 16:07:09.059603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 926/1960] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:09.134827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:09.135817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:09.143481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.143928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.144692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.145121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.174500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-50539fd1-e16a-4bf7-a246-c5bffbb883d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043613 May 23 16:07:09.175043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1035/1961] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:09.384445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:09.384988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:09.391216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.392241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.392995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:09.393432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:09.452779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:09.460659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-104ca419-7142-45e7-a45a-96b7f9d7618b tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080160 May 23 16:07:09.461211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 927/1962] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:10.069795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:10.070754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:10.076783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.077235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.077988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.078423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.106504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-893faccf-33ae-480a-b1d4-250b41913ec0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040669 May 23 16:07:10.107037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1036/1963] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:10.185974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:10.186935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:10.194060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.194539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.202726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.203209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.232469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5341a45f-2e1d-446d-8f7e-2623f3bea424 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050390 May 23 16:07:10.233024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 928/1964] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:10.477414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:10.477956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:10.484430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.484912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.485675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:10.486111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:10.546297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:10.553687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bc5e5a28-1054-4a0e-90f5-8d4eceb70679 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080194 May 23 16:07:10.554245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1037/1965] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:11.117962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:11.118887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:11.124950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.125401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.126725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.127190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.130700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:11.155718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-12c84241-eef0-4002-bf96-19339630bc86 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041706 May 23 16:07:11.156317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 929/1966] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:11.244782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:11.245708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:11.251598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.252041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.252788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.253217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.281434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-221bd55e-fd14-4385-a323-bdc419ee3a41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040650 May 23 16:07:11.281965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1038/1967] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:11.569708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:11.570373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:11.577239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.579379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.579379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:11.579379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:11.638286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:11.646131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9fc1c152-1441-4e76-b914-d632b16af537 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080256 May 23 16:07:11.646693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 930/1968] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:12.167864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:12.168795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:12.175235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.175684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.176439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.176873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.205519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a4531b53-0806-48df-b9e2-f1ce1fef2987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041565 May 23 16:07:12.206073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1039/1969] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:12.293605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:12.294471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:12.300608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.301067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.301890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.302337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.332884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68ed6ca5-1f3d-4d1e-bd9e-5fbb8f999d24 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043354 May 23 16:07:12.333424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 931/1970] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:12.662761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:12.663329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:12.669309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.669754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.670581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:12.671018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:12.733698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:12.741624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d2fab83-7ff3-407c-b9b5-9e76edd3e702 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082948 May 23 16:07:12.742183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1040/1971] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:13.217012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:13.217944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:13.224524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.224976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.225741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.226185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.255347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1d1878c2-53bd-4123-bb57-661d4a2ac9a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042288 May 23 16:07:13.255895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 932/1972] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:13.344242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:13.345225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:13.351629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.352078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.352837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.353270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.539443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f08b5031-9a27-4386-af75-249f5c60795d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.198985 May 23 16:07:13.539993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1041/1973] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:13.757820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:13.758463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:13.764912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.765370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.766133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:13.766593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:13.826688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:13.834218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e55b361-cc94-4a74-837b-4424fca63c82 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080353 May 23 16:07:13.834782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 933/1974] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:13 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:14.266991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:14.267941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:14.273728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.274192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.274942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.275394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.305529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5d606076-6183-459a-be2f-a373645fd6c2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043049 May 23 16:07:14.306143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1042/1975] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:14.550356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:14.551235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:14.557855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.558303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.559062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.559525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.590023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-016dbdf4-7930-4e4f-991d-b1330996e2bd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043386 May 23 16:07:14.590590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 934/1976] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:14.850917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:14.851515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:14.857900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.858349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.859119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:14.859585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:14.918504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:14.926379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae1222a5-1cfb-4eaf-ad27-5903eeb60f55 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079364 May 23 16:07:14.926930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1043/1977] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:15.316258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:15.317196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:15.323288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.323744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:15.324510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.324951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:15.353300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f01c0779-18f8-46b2-9a59-4039033edadf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041030 May 23 16:07:15.353841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 935/1978] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:15.601605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:15.602459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:15.608451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.608904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:15.609667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.610138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:15.637538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-827bc53f-c43d-4fe2-84d8-e91b6bb60598 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039999 May 23 16:07:15.638069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1044/1979] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:15.943943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:15.944498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:15.950997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.958648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:15.958648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:15.958648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:16.012652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:16.020990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9a346b6-bdbc-4e97-a7ea-12040fbadff5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081191 May 23 16:07:16.021550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 936/1980] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:07:16.365396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:16.366333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:16.372743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:16.373195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:16.373950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:16.374384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:16.402681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-399d4c51-6f5d-48cf-aa39-fef97b737378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041282 May 23 16:07:16.403241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1045/1981] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:16.649837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:16.650686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:16.656722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:16.657173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:16.658440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:16.658877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:16.686782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d1c1c40-6b05-4aba-a3de-7b77bd2a0557 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041007 May 23 16:07:16.687354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 937/1982] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:17.038292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:17.038904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:17.044844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.045296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.046055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.046492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.109054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:17.116588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-271e83fe-b79e-4e73-a597-891582aeea51 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082176 May 23 16:07:17.117325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1046/1983] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:17.415382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:17.416320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:17.423194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.424398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.425106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.425548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.456686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b32437b9-aaf3-447d-9ef9-629412145101 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045800 May 23 16:07:17.457271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 938/1984] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:17.697823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:17.698765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:17.705204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.705660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.706431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:17.706873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:17.736674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d6a0e9d-fd00-449e-ab0e-2272acd7918f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042136 May 23 16:07:17.737224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1047/1985] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:18.134002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:18.134641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:18.140856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.141309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.142072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.143199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.209883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:18.217362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f9c667ae-5a1e-44d2-b94b-50ba21a59841 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087216 May 23 16:07:18.217920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 939/1986] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:18.468530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:18.469403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:18.475946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.476393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.477141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.477572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.495977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:18.508161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ed12bda0-6a8b-43e9-b8c9-754540820b66 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043681 May 23 16:07:18.511472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1048/1987] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:18.748123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:18.749051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:18.756294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.756752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.757513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:18.757950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:18.785973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ffe2ee38-d2b6-4629-b1ba-01573d372800 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041799 May 23 16:07:18.786513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 940/1988] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:19.236004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:19.236617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:19.243799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.244360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.245124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.245556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.305503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:19.313164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5f553eb7-43ee-4f87-8abf-3ff8d5914ebb tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081732 May 23 16:07:19.313713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1049/1989] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:19.521180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:19.522144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:19.528156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.528604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.529363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.529801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.557805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dbee0d2e-8485-45f6-ae10-44aca92c9720 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040589 May 23 16:07:19.558343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 941/1990] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:19.797803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:19.798690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:19.805350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.805822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.806584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:19.807034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:19.836394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b6b442f4-d08a-4587-9e38-6fed2443204e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042569 May 23 16:07:19.837142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1050/1991] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:20.330380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:20.330996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:20.338052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.338503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.339307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.339749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.401339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:20.408776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b6f1fc99-25a5-45dd-8bf7-73b69c8d959c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082251 May 23 16:07:20.409339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 942/1992] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:07:20.570032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:20.570966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:20.577445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.577898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.578652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.579090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.607500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c2abe46b-12ff-4963-a234-e10556b2df32 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041884 May 23 16:07:20.608047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1051/1993] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:20.848203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:20.849057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:20.854919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.855398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.856151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:20.856583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:20.885062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d83328cd-46e1-4252-82fb-92eb9f3e5ba0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040850 May 23 16:07:20.885598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 943/1994] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:21.426101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:21.427470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:21.433423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.433869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.434621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.435048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.498017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:21.505490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc3a730f-3ba6-4860-9c62-13dc7d9fc6dd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083270 May 23 16:07:21.506058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1052/1995] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:21.619161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:21.620108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:21.626493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.626943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.627759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.628205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.656882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-44f183a0-9bef-4366-9735-a246c45b18e9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041749 May 23 16:07:21.657417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 944/1996] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:21.897592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:21.898416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:21.904856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.905306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.906059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:21.906490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:21.934028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a200e52b-d92b-471e-b84e-235dfe2e326a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041298 May 23 16:07:21.934563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1053/1997] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:22.523164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:22.523796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:22.532164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.532822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.533935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.534601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.608902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:22.612268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:22.617194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bac504bb-ac50-4bac-89e3-d0d638132145 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.098131 May 23 16:07:22.617746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 945/1998] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:22.667858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:22.668784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:22.674707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.675192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.675956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.676399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.704810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4dc87cf4-104d-493d-b132-8144a0c8c979 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040894 May 23 16:07:22.705341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1054/1999] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:22.946274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:22.947142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:22.954251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.954723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.955506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:22.955952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:22.984083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9dfbff71-5fd5-4059-ac72-cb3b1d18c249 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042558 May 23 16:07:22.984658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 946/2000] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:23.633579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:23.634135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:23.640933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:23.641376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:23.642126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:23.642555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:23.701794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:23.709057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04c42a52-0625-4854-af06-e8da1e63d134 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079468 May 23 16:07:23.710121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1055/2001] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:23.715664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:23.716592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:23.718475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=86 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:23.723144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:23.723598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:23.724361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:23.724875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:23.753059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-444e2034-68e0-41fe-9237-f87f87398aff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041244 May 23 16:07:23.753631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 947/2002] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:23.995547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:23.996475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:24.002492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.002935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.003723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.004155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.032966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f6da54b9-5db6-4831-8b8d-be88caa3f3ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041399 May 23 16:07:24.033538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1056/2003] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:24.727346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:24.727898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:24.734436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.734876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.735674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.736115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.763537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:24.764726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:24.771215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.771672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.772442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:24.772878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:24.796101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:24.802678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e7e4b5a-9614-4bc6-ba7e-5a7eaec76315 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042986 May 23 16:07:24.803267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1057/2004] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:24.803532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d3405685-4ad6-49db-a818-6862f91949b0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081001 May 23 16:07:24.804167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 948/2005] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:25.044866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:25.045836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:25.051972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.052431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.053193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.053635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.083356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-642cd596-dc54-476d-8da6-047df26b4c5d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042426 May 23 16:07:25.083908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1058/2006] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:25.814452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:25.815497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:25.821003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:25.821621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:25.822094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.822550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.823347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.823788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.827879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.828329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.829088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:25.829524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:25.852873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e4d74929-8ac6-4a2b-8e08-e96fa25441fe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042592 May 23 16:07:25.853423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 949/2007] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:25.893399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:25.900899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e6f9d629-8036-4f93-aaed-601c4fa7cacf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084251 May 23 16:07:25.901453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1059/2008] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:25 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:07:26.094647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:26.095554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:26.101517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.101958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.102712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.103167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.131126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c5e0760e-2006-4417-b095-46a5ce1ea5d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040422 May 23 16:07:26.131655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 950/2009] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:26.864455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:26.865385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:26.872624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.873141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.873953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.874391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.904161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-747f9114-c6a2-4861-9370-0cedc9231b15 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043656 May 23 16:07:26.904715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1060/2010] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:26.918223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:26.918843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:26.925321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.925776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.926532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:26.926976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:26.986452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:26.994127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0d0845fe-3bce-4732-a926-abf85e7cba9f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080401 May 23 16:07:26.994788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 951/2011] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:27.143289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:27.144152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:27.150789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:27.151308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:27.152064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:27.152503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:27.180415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de79f50e-d872-4828-937c-766b50df7fe6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041403 May 23 16:07:27.180957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1061/2012] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:27.916663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:27.917610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:27.924048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:27.924612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:27.925382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:27.925831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:27.955172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0729db6e-cefd-4278-b161-4fd7fdb04197 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043328 May 23 16:07:27.955718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 952/2013] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:28.011013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:28.011663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:28.018341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.018791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:28.019628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.020020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:28.079526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:28.087464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e71cb39c-f9d2-4c96-ac60-c57d0f6cb5aa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080293 May 23 16:07:28.088021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1062/2014] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:28.191886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:28.192806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:28.198818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.199309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:28.200063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.200495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:28.229562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6559c393-6912-42b7-871d-40e56f6ace95 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041672 May 23 16:07:28.230119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 953/2015] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:28.966603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:28.967578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:28.969393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=90 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:28.974056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.974540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:28.975322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:28.975757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:29.003981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c9bce4d2-8c59-4ed3-90a1-9347e638ff33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041372 May 23 16:07:29.004551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1063/2016] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:29.104239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:29.104860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:29.111328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:29.111785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:29.112553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:29.112995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:29.173204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:29.181166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5c3d5a9-18d0-4f62-ab38-f1975f05743a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080785 May 23 16:07:29.181726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 954/2017] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:29.240923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:29.241852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:29.249116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:29.249563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:29.250317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:29.250752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:29.279112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca9021b2-24b1-4d45-b264-6dc9fcc2ef7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042169 May 23 16:07:29.279680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1064/2018] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:30.015828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:30.016751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:30.023307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.024320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.025075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.025559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.054368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b426281a-8bab-410e-a7fa-c631e65c817d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042848 May 23 16:07:30.054946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 955/2019] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:30.198422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:30.199033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:30.205097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.205547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.206302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.206737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.210666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:30.272716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:30.281108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dae4284d-38d0-4373-b8fc-fae008a1e122 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087190 May 23 16:07:30.281703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1065/2020] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:30.290924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:30.291895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:30.298262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.298707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.299484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:30.299915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:30.328882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5d00528-9c5d-42ba-b772-fa13e8030072 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041889 May 23 16:07:30.329418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 956/2021] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:31.065849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:31.066791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:31.072877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.073334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.074097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.074546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.103636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fac8d821-377e-4ec8-af24-eb50e5b5eb64 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041720 May 23 16:07:31.104182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1066/2022] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:31.297929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:31.298540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:31.305001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.305446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.306193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.306624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.340315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:31.341176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:31.347815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.348274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.349056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:31.349474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:31.367003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:31.374417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3bab9cff-ca53-43d2-976d-04601f72737d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080333 May 23 16:07:31.374986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 957/2023] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:31.378994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbb1a88a-1384-44f2-9d1f-149723310b9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043092 May 23 16:07:31.379582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1067/2024] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:32.114294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:32.115266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:32.121295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.121742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.122490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.122922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.152798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e96f226-becf-4357-91b6-94984ecb39b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042469 May 23 16:07:32.153340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 958/2025] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:32.390656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:32.391369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:32.391762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:32.391983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:32.398122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.398500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.399226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.399439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.399993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.400192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:32.400435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.400621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:32.428896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4dfc4208-3950-40d3-8f8e-35590b4b06ee tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042372 May 23 16:07:32.429457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1068/2026] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:32.460697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:32.468692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fcc3c725-32dd-40fc-b511-06c0bebb1933 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081502 May 23 16:07:32.469249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 959/2027] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:07:33.164905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:33.165832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:33.171805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.172247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.172989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.173420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.202724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-39685f1f-6115-4097-8421-85b8b5c2f631 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042273 May 23 16:07:33.203286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1069/2028] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:33.440323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:33.441185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:33.447668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.448130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.448899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.449341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.478828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-33c23044-5e02-4be3-b3c1-79d421219168 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042523 May 23 16:07:33.479421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 960/2029] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:33.484606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:33.485324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:33.491888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.492346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.493114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:33.493552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:33.553456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:33.560891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5be289c6-8eb7-434d-bc5e-cc3aad36dbda tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080108 May 23 16:07:33.561443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1070/2030] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:33 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:34.213912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:34.214836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:34.220849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.221294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.222045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.222474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.253215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0bb9418-23fc-4da4-8892-fc49ca3a95f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043251 May 23 16:07:34.253765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 961/2031] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:34.489849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:34.490794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:34.497932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.498379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.499149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.499584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.527311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4a5d89fa-e169-4a85-bdc1-571dcf4410a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041351 May 23 16:07:34.527849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1071/2032] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:34.577377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:34.577982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:34.584567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.585020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.585783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:34.586227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:34.604549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:34.649328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:34.656683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-67d03bc5-16c8-4b54-b508-a773b33843a3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083186 May 23 16:07:34.657238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 962/2033] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:35.265089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:35.266011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:35.272107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.272551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.273293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.273723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.302513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-150509de-de99-47d5-a96f-d1b2aa6f7c63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041666 May 23 16:07:35.303052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1072/2034] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:35.538731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:35.540202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:35.546315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.546764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.547615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.548049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.575982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cdf08fd3-3467-4b46-b791-c9cf808a8996 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041276 May 23 16:07:35.576517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 963/2035] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:35.673074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:35.673690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:35.680101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.680544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.681287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:35.681718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:35.740943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:35.748391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bbf6b396-0a09-491d-9982-30d05e7837ee tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079212 May 23 16:07:35.748944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1073/2036] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:36.314620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:36.315505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:36.322628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.323080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.323885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.324318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.352730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fcb25f47-32ea-4066-98d7-6a358f756594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042148 May 23 16:07:36.353268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 964/2037] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:36.587856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:36.588753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:36.594763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.595252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.596012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.596448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.625818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-882f97ed-03d2-4e75-bd3a-6f9f0354a6b8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042211 May 23 16:07:36.626378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1074/2038] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:36.765462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:36.766078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:36.772106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.772557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.773319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:36.773761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:36.833628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:36.841216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-894c77b5-f928-4f9f-9e41-ece0e542b5ac tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079872 May 23 16:07:36.841730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 965/2039] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:36 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:37.364339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:37.365267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:37.371301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.371753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.372500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.372933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.403307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c602d57c-930a-43a2-bef9-bcb98e89a9a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042838 May 23 16:07:37.403862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1075/2040] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:37.638912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:37.640144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:37.646691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.647172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.647949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.649033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.680831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35c2f8bf-fc8c-4a1e-9294-b38f56aa645e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045943 May 23 16:07:37.681477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 966/2041] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:37.861400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:37.862009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:37.869058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.869412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.870470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:37.870595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:37.933579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:37.941134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a00b4b1a-4cf6-4379-a056-f1e9e04df853 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083604 May 23 16:07:37.941696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1076/2042] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:38.415422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:38.416355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:38.422389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.422840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:38.423633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.424071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:38.452574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb51daa1-75f3-48bc-8a8f-0e8b84893a9a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041628 May 23 16:07:38.453110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 967/2043] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:38.692267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:38.693119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:38.700967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.701648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:38.702858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.703472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:38.732490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8f92bb02-1f71-4b0d-bbe1-89c00208899d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044300 May 23 16:07:38.733040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1077/2044] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:38.959631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:38.960215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:38.966691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.967176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:38.967952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:38.968395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:39.036513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:39.044537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15fcd363-56cc-4e0c-836e-97eae50389a4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.088861 May 23 16:07:39.045100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 968/2045] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:39.465131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:39.466064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:39.472539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:39.472987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:39.473741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:39.474178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:39.504585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb73c268-4068-474c-a325-d5a9e26425ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043907 May 23 16:07:39.505063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1078/2046] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:39.744533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:39.745463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:39.751460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:39.751911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:39.752669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:39.753116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:39.781847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de6646d6-2f1e-496d-a2f6-c12c58fe88d4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042065 May 23 16:07:39.782401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 969/2047] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:40.061970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:40.062580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:40.068601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.069046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.069797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.070225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.128809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:40.136273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0301280d-0712-486b-9c0c-34b116d7e241 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078198 May 23 16:07:40.136823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1079/2048] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:40.516303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:40.517238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:40.523724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.524798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.525558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.525996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.554331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b17130d-fdf6-4866-bc62-1f7a3ca057ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042200 May 23 16:07:40.554888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 970/2049] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:40.793040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:40.793917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:40.800411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.800858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.801614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:40.802039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:40.830820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d62ed4e0-e06d-47f6-97fa-293e7343a687 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041803 May 23 16:07:40.831386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1080/2050] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:41.153489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:41.154025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:41.160075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.160529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.161326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.161771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.222666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:41.230700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-158feec5-44ec-4df9-8bbd-5b4486b3100a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081175 May 23 16:07:41.231289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 971/2051] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:41.565963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:41.566899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:41.573015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.573477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.574230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.574670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.599065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:41.604132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d34f914d-c53a-4635-bf81-a77e67dc7c81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042279 May 23 16:07:41.604681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1081/2052] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:41.842796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:41.843786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:41.850309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.850758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.851550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:41.851985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:41.881139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bcedc034-9edc-4015-a787-c822cf8e8f6f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042401 May 23 16:07:41.881679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 972/2053] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:42.247068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:42.247789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:42.254625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.255070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.255834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.256266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.320493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:42.328441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b0bd335f-0d66-49f1-ac87-b633df29fe9a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.085197 May 23 16:07:42.329011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1082/2054] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:42.616119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:42.617039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:42.623516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.624005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.624758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.625190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.654042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd5244ac-670a-4229-9bc5-e8f48a59275f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041890 May 23 16:07:42.654589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 973/2055] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:42.892964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:42.893826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:42.899853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.900308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.901067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:42.901507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:42.929767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-62fbf6bb-e2a0-4a4b-9095-f3e7173db6f2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041102 May 23 16:07:42.930923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1083/2056] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:43.345742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:43.346360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:43.353394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.353846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.354598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.355035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.417853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:43.425731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-134a24fa-978b-46a6-b475-715b7f503ce1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083860 May 23 16:07:43.426493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 974/2057] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:43.665399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:43.666322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:43.673142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.673647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.674405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.674844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.705020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad9d94d5-809f-43d3-affd-f223f4c62258 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043509 May 23 16:07:43.705570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1084/2058] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:43.941842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:43.942723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:43.949580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.950031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.950795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:43.951271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:43.979173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-170bfa81-769f-4a8f-91b5-edcdeb44e527 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041424 May 23 16:07:43.979717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 975/2059] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:44.441744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:44.442356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:44.448456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.448912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:44.449672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.450112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:44.509671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:44.517630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-43389c12-9bfd-459a-9d93-1a968774371e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079758 May 23 16:07:44.518199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1085/2060] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:44.716616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:44.717540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:44.724171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.724617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:44.725370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.725801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:44.755485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8b435a64-fbf2-48fa-80ca-6a0de3f719ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042750 May 23 16:07:44.756031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 976/2061] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:44.989891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:44.990883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:44.997987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.998437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:44.999229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:44.999666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:45.027851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ece4a5b-a124-4bb7-8073-297aab0da60e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041991 May 23 16:07:45.028399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1086/2062] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:45.533874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:45.534499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:45.540618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:45.541075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:45.541872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:45.542314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:45.602443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:45.609873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0db6dda-b245-406c-90ef-d5d86b640d2a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080031 May 23 16:07:45.610429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 977/2063] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:45.767567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:45.768496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:45.774871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:45.775350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:45.776097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:45.776528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:45.806108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-06f2a1c8-31e6-4a2a-8c9a-66abc3e7b5ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042427 May 23 16:07:45.806585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1087/2064] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:46.040150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:46.041107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:46.047544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.047991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.048750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.049181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.077475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f5969081-5fac-4087-bfa6-fdf0d8d03759 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041936 May 23 16:07:46.078014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 978/2065] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:46.627347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:46.628001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:46.635118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.635598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.636357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.636836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.697564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:46.705062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9d779fce-f012-4b83-a9d3-17695663950e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081631 May 23 16:07:46.705620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1088/2066] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:46.817688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:46.818619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:46.825209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.825689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.826447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:46.826883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:46.830439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:46.856343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-010a9cf3-119a-4aab-8584-4fd9c445db41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042729 May 23 16:07:46.856882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 979/2067] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:47.088845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:47.089765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:47.095790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.096235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.096978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.097406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.125788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f58346b0-3e5d-4e33-b154-a44b7911863d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040889 May 23 16:07:47.126336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1089/2068] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:47.722581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:47.723168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:47.730595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.731047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.731833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.732272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.794073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:47.801705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc088a00-d56c-4d81-83b8-f93aede90a48 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083118 May 23 16:07:47.802293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 980/2069] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:47.867843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:47.868786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:47.875312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.875770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.876530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:47.876970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:47.905636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f80437cf-45b6-415e-a32f-4bf8c728810b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041778 May 23 16:07:47.906177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1090/2070] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:48.137801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:48.138728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:48.144931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.145376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.146129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.146615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.175011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a04dfec-641c-4ba1-8ade-6412898a4b03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041681 May 23 16:07:48.175582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 981/2071] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:48.818551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:48.819200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:48.825208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.825657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.826409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.826841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.886271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:48.893788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de8d35d5-a3fd-489b-8f27-37a2647636f4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079104 May 23 16:07:48.894340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1091/2072] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:48 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:48.916486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:48.917403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:48.923786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.924231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.924984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:48.925414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:48.954648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdab55dd-c941-4019-83cd-7da986513cbc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042477 May 23 16:07:48.955220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 982/2073] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:49.186713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:49.187617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:49.193557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.193991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.194741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.195205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.223342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-10cd4874-8dd3-4a3e-bbee-31d7535e5569 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040930 May 23 16:07:49.223883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1092/2074] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:49.910259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:49.910899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:49.917584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.918308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.919528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.920281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.965815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:49.966735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:49.973095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.973548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.974294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:49.974731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:49.980966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:49.988304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e0f8659-4e74-4644-96cd-4175baf76945 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081873 May 23 16:07:49.988864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 983/2075] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:50.004221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e941d38-4d46-4255-8d93-d4d98f52e0ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042253 May 23 16:07:50.004782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1093/2076] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:50.236410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:50.237358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:50.244141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:50.244594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:50.245355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:50.245786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:50.274150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a34b679d-6217-4dcc-b7af-634d5ac018c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042605 May 23 16:07:50.274703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 984/2077] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:51.007796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:51.008900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:51.015664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.015819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:51.016216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.016717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:51.016831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.017255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.023067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.023555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.024908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.025347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.054425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8f2f7dea-5af5-4f47-92d3-d070848248f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042834 May 23 16:07:51.054989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 985/2078] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:51.080580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:51.088689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aae470cf-1369-44bf-9167-4500cf90c109 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084991 May 23 16:07:51.089236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1094/2079] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:51.285996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:51.286859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:51.292880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.293333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.294092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:51.294554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:51.322765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fddc1632-fe26-459a-a0d5-9fdf00518c7c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040870 May 23 16:07:51.323399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 986/2080] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:52.066264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:52.067256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:52.073724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.074178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.074940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.075410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.104420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14a98c3e-e96e-44d2-b299-87018717e262 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042392 May 23 16:07:52.104978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1095/2081] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:52.105773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:52.106389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:52.112738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.113187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.113940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.114371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.176670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:52.184307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-afd5c493-7c3b-4a1d-bb66-75becf9f9d76 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082917 May 23 16:07:52.184869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 987/2082] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:07:52.333899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:52.334833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:52.341350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.341807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.342564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:52.343004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:52.375553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da02c62d-61b4-427f-9f68-4141dbe97711 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045560 May 23 16:07:52.376110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1096/2083] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:53.115993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:53.116954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:53.123915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.124359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.125120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.125553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.153923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39e6de39-c75d-4b17-8856-fcb2ac04788a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041908 May 23 16:07:53.154471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 988/2084] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:53.204979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:53.205598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:53.212053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.212493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.213241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.213672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.273600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:53.281011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf1d33c9-4226-4abe-bbd7-65f69aae2ef6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082881 May 23 16:07:53.281563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1097/2085] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:53.387174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:53.388032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:53.394318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.394805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.395602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:53.396039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:53.428515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0bbee3cb-20df-43c7-b910-518e0e055be8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045316 May 23 16:07:53.431596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 989/2086] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:54.164823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:54.165826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:54.172029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.172488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.173246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.173685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.190694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:54.203309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1c850c4-8d8e-4137-9c54-88f3e393e0a5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042567 May 23 16:07:54.203861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1098/2087] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:54.298287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:54.298907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:54.305387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.305836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.306599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.307033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.366215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:54.374275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6f9da08-b364-401a-baf6-4a1b85590f13 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079888 May 23 16:07:54.374834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 990/2088] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:54.441005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:54.441962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:54.449355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.449811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.450565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:54.451003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:54.479254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1e708342-7b27-4fd2-bc57-60459c191558 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042275 May 23 16:07:54.479817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1099/2089] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:55.216846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:55.217778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:55.219721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=88 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:55.224419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.224866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.225622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.226060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.257469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-048dfbdc-7851-448e-a73d-4519e2738453 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044710 May 23 16:07:55.258074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 991/2090] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:55.391144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:55.391832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:55.398192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.398637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.399955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.400384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.459851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:55.467834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-006bd4a8-0b26-4314-a65d-b85938ba0593 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080864 May 23 16:07:55.468389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1100/2091] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:55.489751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:55.490678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:55.496707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.497160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.497928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:55.498368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:55.528495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e02479e-a410-4c81-b024-7844e0e36a33 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042584 May 23 16:07:55.528972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 992/2092] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:56.269244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:56.270188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:56.276269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.276713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.277460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.277890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.307245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-701e2c3b-4185-422c-a096-937ce37e5c45 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041877 May 23 16:07:56.307786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1101/2093] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:56.484486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:56.485108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:56.491708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.492263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.493572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.494009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.557095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:56.564663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c7ac2ffc-0f22-463e-91ff-c8fda50ffb0d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084192 May 23 16:07:56.565226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 993/2094] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:56.626341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:56.627283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:56.633133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.633575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.634323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:56.634753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:56.663070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c4ccbc5a-211d-474b-a5e8-9ef4fca0a9c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.127186 May 23 16:07:56.663629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1102/2095] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:57.318718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:57.319676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:57.326105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.326568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.327326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.327764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.355576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3593f8c0-c944-477c-b568-2e39572d2fb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040766 May 23 16:07:57.356106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 994/2096] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:57.581326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:57.581875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:57.588961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.589416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.590177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.590617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.650034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:57.657458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c17fe5fa-5a78-4255-a3a9-830e98c285d6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080016 May 23 16:07:57.658014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1103/2097] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:57.675209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:57.676131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:57.682430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.682872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.683667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:57.684105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:57.708141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:57.713330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b9d7ac09-0c99-4438-8f8a-05d68e400dad tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042059 May 23 16:07:57.713874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 995/2098] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:58.367455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:58.368311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:58.377887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.378341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.379123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.379559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.408335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4cc3c16-f0e9-4705-822b-1b470ffa38c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045050 May 23 16:07:58.408884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1104/2099] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:58.675558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:58.676100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:58.682508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.682956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.683743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.684177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.729831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:58.730748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:58.737064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.737517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.738261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:58.738695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:58.743800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:58.751322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-14253bd7-b74a-4ee0-ba6a-a1fb0bd8cd26 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079990 May 23 16:07:58.752083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 996/2100] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:58 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:07:58.767668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bea65b6f-697f-4bb4-9fcc-a2b7a5ce32d0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041681 May 23 16:07:58.768207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1105/2101] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:07:59.419611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:59.420547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:59.426567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.427015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.427804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.428237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.456657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d8de939c-0165-49b7-aaf7-1d2a41e3bcbe tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041105 May 23 16:07:59.457193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 997/2102] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:59.768967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:59.769573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:59.771926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:59.777273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.777722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.778473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.778902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.779787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:07:59.780707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:07:59.787193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.790279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.790279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:07:59.790279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:07:59.819381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57111dfc-57d0-4e51-b56b-14c897c60f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044235 May 23 16:07:59.819950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 998/2103] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:07:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:07:59.840339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:07:59.847877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6feab3ed-1b52-4895-958a-7a6ad5165f30 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083124 May 23 16:07:59.849135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1106/2104] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:07:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:00.468895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:00.469749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:00.475856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.476313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.477116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.477560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.505447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-98d182d9-c5ab-447f-8268-58ca27af022d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040723 May 23 16:08:00.505978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 999/2105] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:00.830238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:00.831220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:00.837632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.838083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.841280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.841280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.865259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:00.865899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:00.867335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f84656c8-cf88-489f-8dfb-06e3d59a2486 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041014 May 23 16:08:00.868390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1107/2106] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:00.872811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.873254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.874010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:00.874440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:00.936705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:00.944090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b7de5cb-ed9c-4c1f-ab80-2b5e64ab5ac4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082705 May 23 16:08:00.944651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1000/2107] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:01.517379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:01.518240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:01.524219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.524700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:01.525458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.525900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:01.556325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9a4cee15-d6f7-41a6-b979-c41d357149e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043325 May 23 16:08:01.556874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1108/2108] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:01.879609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:01.880443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:01.887452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.887898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:01.888650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.889080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:01.918679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-497c274a-b0a3-4e23-8eb8-7939eea72cb2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043204 May 23 16:08:01.919245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1001/2109] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:01.962376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:01.963183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:01.969539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.969986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:01.970733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:01.971183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:02.031251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:02.038846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5cbdeddb-d569-4a61-8ee9-0142e0a73f22 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081446 May 23 16:08:02.039438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1109/2110] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:01 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:02.568089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:02.568940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:02.575259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:02.575707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:02.576467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:02.576899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:02.605609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ab93f11b-dddb-4998-b97c-5aaae1d00ba5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041513 May 23 16:08:02.606162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1002/2111] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:02.930103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:02.930930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:02.937438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:02.937884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:02.938626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:02.939058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:02.967851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-25e23c00-e6d2-4d48-9b79-bb671de2116b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041788 May 23 16:08:02.968587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1110/2112] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:03.055326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:03.055937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:03.062294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.062744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:03.063528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.063962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:03.128122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:03.135700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c7a08502-064f-4458-a220-9cbb649fe82e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084203 May 23 16:08:03.136254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1003/2113] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:03 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:03.617447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:03.618292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:03.624217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.624661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:03.625410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.625842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:03.654583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1df2724-29bc-4d22-8295-13b4e472e5ab tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041205 May 23 16:08:03.655139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1111/2114] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:03.979531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:03.980396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:03.986380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.986826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:03.987626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:03.988064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:04.016495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c00aac0a-2745-4365-9c74-86f90e48222e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041576 May 23 16:08:04.017036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1004/2115] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:04.154051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:04.154661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:04.161060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:04.161500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:04.162244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:04.162673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:04.221719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:04.229085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a33b28f7-9014-43ac-8c71-c1e3cf38d7f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079493 May 23 16:08:04.229701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1112/2116] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:04 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:04.666449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:04.667392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:04.673731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:04.674186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:04.674924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:04.675382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:04.704299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d87925f9-67b0-4880-8eee-d9eae7b910b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041828 May 23 16:08:04.705030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1005/2117] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:05.028532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:05.029462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:05.035484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.035939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.036700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.037133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.065339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-895d3e03-d406-44fd-9b6d-8fccd5f23e56 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041312 May 23 16:08:05.065888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1113/2118] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:05.245865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:05.246490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:05.252609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.253064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.253829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.254266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.316842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:05.324572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15e23eef-0447-4dea-aeb9-fde898366bff tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082668 May 23 16:08:05.325174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1006/2119] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:05 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:05.716366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:05.717213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:05.724277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.724724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.725467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:05.725897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:05.755574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e658bbab-731f-480f-94bb-7f5ea5c431ce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043744 May 23 16:08:05.756108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1114/2120] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:06.076867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:06.077850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:06.084287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.084732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.085910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.116137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c57a63cb-53d1-49ca-9e4c-50e7e434359d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043308 May 23 16:08:06.116677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1007/2121] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:06.341540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:06.342080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:06.349028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.349480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.350234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.350668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.354839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:06.410610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:06.418332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db8aafee-c845-4a8d-8df2-d3a948bf8244 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080757 May 23 16:08:06.418904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1115/2122] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:06 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:06.766778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:06.767742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:06.773805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.774262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.775561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:06.776006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:06.804811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e105845d-0e45-4cb1-984c-869ae628f27b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042288 May 23 16:08:06.805355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1008/2123] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:07.128306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:07.129234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:07.135316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.135771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.136563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.136996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.164860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6027c562-0431-4177-9c85-d82b8e472ed5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040542 May 23 16:08:07.165391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1116/2124] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:07.435692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:07.436255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:07.442729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.443222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.443987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.444434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.506662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:07.512704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-32f8a6fd-6b0d-4da3-8a41-649223eb2717 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081483 May 23 16:08:07.513271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1009/2125] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:07 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:07.816175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:07.817127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:07.824100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.824549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.825337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:07.825773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:07.855382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6c086d66-a12c-45ae-a51a-51050faa7b28 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043138 May 23 16:08:07.855923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1117/2126] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:08.175824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:08.176751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:08.183239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.183683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.184436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.184866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.220287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5777db13-785b-4884-974e-bdbad967c3c9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048276 May 23 16:08:08.220846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1010/2127] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:08.530399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:08.531013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:08.536991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.537434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.538183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.538616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.614374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:08.621948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-23d1f90e-72f7-4d48-911e-2c1821fe7f41 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.095470 May 23 16:08:08.622495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1118/2128] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:08 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:08.866642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:08.867520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:08.874690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.875169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.875928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:08.876367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:08.905375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-21bb1153-f182-481f-9b8a-1fcdaea4712f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042784 May 23 16:08:08.905916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1011/2129] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:09.232456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:09.233404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:09.239682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.240135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.240900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.241334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.279358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9a16675-7b9d-45be-a3c8-31cc2bda7080 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051357 May 23 16:08:09.279916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1119/2130] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:09.638688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:09.639280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:09.645577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.646029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.646785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.647261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.665576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:09.710843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:09.718811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3805e764-da36-4a75-83a0-b20b468e263f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084794 May 23 16:08:09.719398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1012/2131] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:09 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:09.917574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:09.918430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:09.924402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.924858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.925620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:09.926062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:09.959428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7c4e0945-f4ab-4074-b9f1-38f871246ea7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044088 May 23 16:08:09.960019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1120/2132] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:10.290966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:10.291936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:10.298518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.298974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:10.299783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.300222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:10.335917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d49eced0-108b-4824-b0d7-9c12d241e19c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048900 May 23 16:08:10.336468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1013/2133] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:10.736257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:10.741492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:10.743364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.743820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:10.744580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.745016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:10.805371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:10.814947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-26760529-f029-4d8c-bcc5-2dd3080ab5a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082573 May 23 16:08:10.815549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1121/2134] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:10 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:10.971539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:10.972458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:10.978379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.978824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:10.979644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:10.980080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:11.007585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b97bc88-d716-4e29-8448-ec3671bd207f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039955 May 23 16:08:11.008058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1014/2135] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:11.348191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:11.349049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:11.354998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:11.355480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:11.356228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:11.356661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:11.385216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ebe66774-bcec-4384-b549-3ea6179d3a72 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040385 May 23 16:08:11.385750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1122/2136] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:11.832275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:11.832899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:11.839394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:11.839852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:11.840615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:11.841050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:11.901409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:11.908846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb067b5a-7788-4c9c-8652-fbb50f5691be tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080457 May 23 16:08:11.909401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1015/2137] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:11 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:12.019563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:12.020491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:12.027410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.027868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.028623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.029065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.058555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da80b0c4-c032-451e-8d08-122079eaaeb3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043002 May 23 16:08:12.059118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1123/2138] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:12.396882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:12.397807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:12.404426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.404872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.405625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.406058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.438924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70b99a24-85c7-4000-ad6b-4bb5d61b5462 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046013 May 23 16:08:12.439503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1016/2139] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:12.925883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:12.926523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:12.932413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.932853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.933595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:12.934025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:12.992924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:13.000292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db82973c-4e7e-4b1b-917d-dc8131b1ab2e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078244 May 23 16:08:13.000866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1124/2140] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:12 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:13.070084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:13.071046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:13.077970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:13.078422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:13.079242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:13.079685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:13.107931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-345d68cb-f0ea-426b-b929-5275ae87c701 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041762 May 23 16:08:13.108480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1017/2141] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:13.450378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:13.451338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:13.457726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:13.458171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:13.458920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:13.459384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:13.487735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-796a03d8-c12a-4674-b5b7-ea9de308784c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041851 May 23 16:08:13.488196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1125/2142] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:14.017305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:14.017917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:14.023963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.024408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.025826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.026260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.088670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:14.096146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7dec80d7-df03-468b-9320-2e4edf50e3ad tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083059 May 23 16:08:14.096704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1018/2143] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:14 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:14.119391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:14.120342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:14.126280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.126726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.127515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.127946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.157184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7be7f02-e5aa-469f-b716-eaafc8714069 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041766 May 23 16:08:14.157720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1126/2144] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:14.499547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:14.500513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:14.506526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.506979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.507764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:14.508206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:14.537284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c2d7a4f2-4873-41ca-a094-70dfe8a3a7d7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041724 May 23 16:08:14.537819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1019/2145] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:15.112940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:15.113564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:15.120413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.122206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.122206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.122206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.168801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:15.169732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:15.176131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.176580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.177346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.177786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.185359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:15.191552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3db2bf7b-b5df-4d80-9ad9-5bc5f6176d4d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082435 May 23 16:08:15.192174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1127/2146] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:15 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:15.208153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70c1ccec-f494-402f-b6e3-d4fb8559b138 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043364 May 23 16:08:15.208708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1020/2147] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:15.549441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:15.550335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:15.556912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.557366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.558129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:15.558568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:15.586159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b86c733a-0658-4e06-ac63-ee1edaf17c22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040734 May 23 16:08:15.586707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1128/2148] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:16.208862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:16.209479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:16.216647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.217038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.217801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.218239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.219332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:16.220282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:16.226542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.226987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.227822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.228259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.257376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-720d46ff-9213-4ca9-ad6c-a8a4f911d870 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041925 May 23 16:08:16.257933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1129/2149] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:16.281607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:16.289081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bd3225bf-dbb8-42a5-93dc-c144b45fd4a5 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084125 May 23 16:08:16.289640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1021/2150] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:16 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:16.597845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:16.599089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:16.605764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.606210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.606988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:16.607443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:16.635079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3ca6e52e-6e36-4440-8a67-808a2b06fea6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041231 May 23 16:08:16.635638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1130/2151] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:17.269229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:17.270167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:17.277280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.277733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.278494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.278930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.306115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:17.306860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:17.307339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7eb1c2d-5366-436f-adaa-106a98f3f9ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042269 May 23 16:08:17.307898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1022/2152] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:17.313287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.313744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.314509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.314949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.374901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:17.378347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:17.382668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e077b0f3-82a1-4aa2-8746-371a21304e8c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080392 May 23 16:08:17.383269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1131/2153] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:17 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:17.646242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:17.647115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:17.653633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.654084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.654847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:17.655320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:17.682979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9199ddc9-5eac-4871-93de-c2002fc6b84d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040801 May 23 16:08:17.683565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1023/2154] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:17.992924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:08:17.993154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:08:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:08:18.318512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:18.319468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:18.326495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.326938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.327729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.328162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.358913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e514b7f-81a8-46e3-9707-cfd7cefe3be7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044647 May 23 16:08:18.359518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1132/2155] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:18.398993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:18.400607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:18.407037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.407518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.408272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.408711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.471783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:18.479799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e9a5cc9-b548-4a56-9550-797d63f3067c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084621 May 23 16:08:18.480375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1024/2156] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:18 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:18.693783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:18.694699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:18.701327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.701777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.702524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:18.702952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:18.731655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8d7cbc27-8e6c-49e3-a5ad-15a8b32915f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041763 May 23 16:08:18.732194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1133/2157] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:19.369880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:19.370813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:19.377037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.377489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.378243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.378676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.407517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-137a02c6-41fb-4fc0-bdcb-20f67eab4a13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041608 May 23 16:08:19.408086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1025/2158] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:19.496936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:19.497484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:19.503947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.504395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.505141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.505575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.565119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:19.573091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-656a18fd-e4be-4fa6-8f43-3170e0184ad4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080588 May 23 16:08:19.573685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1134/2159] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:19 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:19.743256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:19.744190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:19.753420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.753420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.753420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:19.753420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:19.781197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08187dfa-9019-4755-9818-c4dda7034ed9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041852 May 23 16:08:19.782140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1026/2160] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:20.419015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:20.419937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:20.425961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.426413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.427205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.427648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.456556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-26e4fb82-2f5a-47b0-8388-e5a117343ae8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041590 May 23 16:08:20.457097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1135/2161] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:20.589467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:20.590109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:20.596522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.597009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.597767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.598204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.657483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:20.664841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9ee3e38-d020-4a54-a0d8-09743b070acd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079259 May 23 16:08:20.665935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1027/2162] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:20 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:20.792923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:20.793855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:20.800341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.800792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.805231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:20.805231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:20.831046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-024dd70b-d77e-4ba9-bf1d-db8c8e0346b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042086 May 23 16:08:20.831614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1136/2163] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:21.468224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:21.469151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:21.475087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.485721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.485721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.485721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.505448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-29e514ec-fd2f-4553-9d36-718d09fcfbfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041336 May 23 16:08:21.505986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1028/2164] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:21.682370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:21.682980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:21.689092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.689541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.690297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.690731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.751053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:21.758511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17374fff-66a2-4ff2-8b8d-e7c16eaf337f tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080626 May 23 16:08:21.759150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1137/2165] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:21 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:21.842677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:21.843639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:21.850047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.850498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.851288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:21.851724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:21.855859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:21.880859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bd369756-640d-4ce6-9359-7f860f7fb4a3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042291 May 23 16:08:21.881399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1029/2166] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:22.517470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:22.518412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:22.524953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.525412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.526171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.526611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.554606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-854970a1-44ef-4410-a1af-f457d097ee5a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041407 May 23 16:08:22.555179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1138/2167] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:22.776178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:22.777304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:22.783480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.783934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.784694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.785129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.847291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:22.855241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fd8a58d1-24e8-4b72-a4bd-2202558ac2bf tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083379 May 23 16:08:22.855796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1030/2168] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:22 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:22.893303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:22.894265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:22.900284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.900744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.901503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:22.902514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:22.930228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2060ef8f-1d2d-45e1-bba7-7c31c0bc98a8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041090 May 23 16:08:22.930774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1139/2169] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:23.567159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:23.568053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:23.574518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.574962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.575744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.576179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.606777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3fb5174a-1c7a-4656-8af6-f53b76113e63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043975 May 23 16:08:23.607344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1031/2170] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:23.872080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:23.872611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:23.878976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.879465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.880237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.880665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.940345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:23.941711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:23.942639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:23.948078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d2e4d9ba-745d-4172-8af2-1c3bdd208ce3 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080610 May 23 16:08:23.949487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.949935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.950247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1140/2171] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:23 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:23.950762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:23.951249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:23.979075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0448477-dc98-4626-a8aa-12f6d5605e5e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041251 May 23 16:08:23.979636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1032/2172] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:24.617726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:24.618585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:24.625229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.625684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:24.626445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.626887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:24.655485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89930b1e-2049-41cb-ab82-b14c94e908a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041800 May 23 16:08:24.656037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1141/2173] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:24.965186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:24.965710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:24.972160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.972610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:24.973370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.973811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:24.989870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:24.991295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:24.997866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.998327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:24.999114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:24.999562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:25.035719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2640985f-106d-4d25-8625-6406aa66574c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049700 May 23 16:08:25.036014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:25.036295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1142/2174] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:25.044199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6076131a-8e05-4647-9ff9-d94deeab899a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083074 May 23 16:08:25.044753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1033/2175] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:24 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:25.668147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:25.668987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:25.675016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:25.675496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:25.676246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:25.676676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:25.706378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f5b8777-2403-4140-8408-3495b984b635 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042758 May 23 16:08:25.706933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1143/2176] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:26.047703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:26.048634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:26.050915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=86 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:26.055675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.056128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.056933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.057370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.063243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:26.071308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:26.071308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.071308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.071642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.071805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.088088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cbcd9c97-e6cf-40a8-a7eb-fcf8888cc9f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044467 May 23 16:08:26.088649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1034/2177] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:26.132178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:26.140453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e06d324a-7118-47e7-9cab-4c16fddf3d8e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080836 May 23 16:08:26.141008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1144/2178] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:26 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:26.719132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:26.720054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:26.726180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.726635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.727425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:26.727861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:26.755266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-16efe70a-fd3b-411e-b9ce-0e2b0498e18a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040961 May 23 16:08:26.755798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1035/2179] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:27.100040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:27.100990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:27.107529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.107977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.108726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.109159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.137422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c8dc080a-7066-4220-a01b-f5b1d957a79d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041323 May 23 16:08:27.137948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1145/2180] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:27.156845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:27.157455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:27.163857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.164302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.165053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.165487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.227822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:27.236022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ef031cf-55d9-45b3-aa0b-16b8673d0fa6 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082876 May 23 16:08:27.236586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1036/2181] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:27 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:27.766839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:27.767786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:27.773722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.774164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.774903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:27.775369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:27.804104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db9b7c9d-0997-4722-8631-6b2c1f62fda4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041478 May 23 16:08:27.804643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1146/2182] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:28.149062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:28.149989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:28.156648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.157096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.157878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.158310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.185670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c7253543-27e9-450a-86fc-52261bb346c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040574 May 23 16:08:28.186205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1037/2183] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:28.252675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:28.253294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:28.259718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.260169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.261437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.261875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.322869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:28.330892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-904a1635-9428-4989-abd9-b80bcc60bc79 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082105 May 23 16:08:28.331537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1147/2184] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:28 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:28.815487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:28.816422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:28.822393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.822846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.823652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:28.824089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:28.852209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35ca25a1-0d72-49c9-8b10-eb1b34d15233 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040696 May 23 16:08:28.852751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1038/2185] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:29.197126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:29.197998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:29.204141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.204579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.205337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.205779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.222281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:29.235273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c902c0b4-95e8-4a25-ab31-ca261eadde7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042139 May 23 16:08:29.239915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1148/2186] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:29.347611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:29.348232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:29.354562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.355674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.356438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.356882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.415408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:29.424153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ff7e76be-f053-470a-9669-618440e1d5de tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080319 May 23 16:08:29.424718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1039/2187] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:29 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:29.864134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:29.865066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:29.867688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=86 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:29.873139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.873596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.874353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:29.874791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:29.903657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-07ef7dec-adb5-4fcc-a302-0cdecbaa03ee tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043461 May 23 16:08:29.904214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1149/2188] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:30.249224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:30.250163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:30.256367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.256820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.257585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.258021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.288007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd9b7533-a236-4f79-9015-2ce4bd2eac7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043312 May 23 16:08:30.288539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1040/2189] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:30.441229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:30.441836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:30.447731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.448176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.448933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.449360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.512545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:30.520376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5cf995d9-8d45-4075-9edf-a3e8a2878fa4 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083388 May 23 16:08:30.520930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1150/2190] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:30 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:30.914649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:30.915518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:30.922499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.922947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.923738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:30.924176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:30.951812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f349a562-5b93-4cc6-a1e4-7ad22a9a7582 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041239 May 23 16:08:30.952342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1041/2191] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:31.299657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:31.300591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:31.307007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.307490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:31.308850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.309283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:31.337290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-00a26705-0a96-4522-a198-70108f62c8c0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041601 May 23 16:08:31.337824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1151/2192] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:31.537114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:31.537644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:31.544188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.544633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:31.545427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.545857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:31.609098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:31.616510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef91e90e-81b1-4f73-a655-7ce729b8b41e tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083382 May 23 16:08:31.617065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1042/2193] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:31 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:31.963723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:31.965310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:31.971209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.971664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:31.972415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:31.972845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:32.000955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c2eea61f-2336-495a-9d50-b7179a8c3b27 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041363 May 23 16:08:32.001511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1152/2194] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:32.348767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:32.349702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:32.356090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:32.356539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:32.357294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:32.357730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:32.386659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e136a5a-5448-470e-9768-83f83a082b22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041295 May 23 16:08:32.387229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1043/2195] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:32.633150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:32.633689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:32.640894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:32.641343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:32.642128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:32.642566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:32.702772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:32.710318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2cd042d-0ce4-48ca-8abb-ae1fb1d45fdc tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081228 May 23 16:08:32.710880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1153/2196] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:32 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:33.012339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:33.013270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:33.019579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.020035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.020799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.021238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.050053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c738c05f-246d-45ec-bde7-1a292d8f7abc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041686 May 23 16:08:33.050594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1044/2197] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:33.397977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:33.398839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:33.405434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.405884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.406633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.407069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.434495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-efbd8a37-61c8-4097-b9a4-8768fcacdc8e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040694 May 23 16:08:33.435032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1154/2198] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:33.727269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:33.727804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:33.734412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.734865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.735655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:33.736093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:33.802734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:33.806106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:33.811005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fb728ac4-0940-4d67-ba5d-0842b7d69d38 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087753 May 23 16:08:33.811599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1045/2199] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:33 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:34.062184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:34.063121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:34.069557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.070270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.071013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.071482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.100441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a4c7836-9b54-4797-b592-43a7f1ccaea0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042817 May 23 16:08:34.100996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1155/2200] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:34.446043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:34.446888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:34.453287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.453736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.454486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.454933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.483895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d41038c-3cd2-433a-9658-56fc8fd23f3c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041850 May 23 16:08:34.484450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1046/2201] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:34.828372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:34.828990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:34.835189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.835649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.836406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:34.836842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:34.896706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:34.904646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-98efc9bd-658d-452c-b726-1a55caba4a29 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080253 May 23 16:08:34.905220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1156/2202] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:34 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:35.111491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:35.112335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:35.119032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.121066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.121066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.121353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.150785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d335a954-4260-4cf5-bdbe-32914d218884 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043357 May 23 16:08:35.151358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1047/2203] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:35.496126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:35.497062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:35.503566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.504022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.504779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.505216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.535310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5622232c-e4ab-4662-861f-999496b3bd05 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043775 May 23 16:08:35.535858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1157/2204] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:35.921675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:35.922220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:35.928259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.928712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.929464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:35.929900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:35.957651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:08:35.957841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:08:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:08:35.991862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:36.000073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53398671-ce4c-4c88-8d1d-548d2c8ed5f8 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082310 May 23 16:08:36.000638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1048/2205] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:35 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:36.162255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:36.163250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:36.169440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:36.169897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:36.170652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:36.171116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:36.200383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a5e4a7b-3d20-44d5-9d86-94e899a328c1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042026 May 23 16:08:36.200855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1158/2206] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:36.547755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:36.548687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:36.555107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:36.555575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:36.556322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:36.556760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:36.584930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3e8eac44-068e-464b-999a-7ca0d705d3f3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041162 May 23 16:08:36.585466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1049/2207] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:37.017510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:37.018126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:37.024110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.024559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.025307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.025746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.085115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:37.093065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c304cf6b-c9d0-4875-ac0b-35f13d8d3eea tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079678 May 23 16:08:37.093620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1159/2208] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:37 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:37.211895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:37.212816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:37.219441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.219891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.220969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.221405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.249461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99c552d7-6592-4807-8e75-be75db843e05 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041397 May 23 16:08:37.250008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1050/2209] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:37.596423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:37.597356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:37.603932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.604390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.605161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:37.605595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:37.634083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-15a24c9e-e934-43f2-9294-9e24c1165ad2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041716 May 23 16:08:37.634632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1160/2210] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:38.110362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:38.110982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:38.117668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.118113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.118865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.119324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.181302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:38.188837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52759f3b-b9fa-4898-8352-ba9863c4732c tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082485 May 23 16:08:38.189397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1051/2211] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:38 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:38.260805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:38.261763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:38.268269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.268719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.269473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.269905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.299373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1eb3700f-ee3e-435f-ae08-8ad726c55223 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042451 May 23 16:08:38.299929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1161/2212] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:38.646532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:38.647479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:38.654032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.654477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.655255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:38.655690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:38.682811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-beb3d16c-955b-4f46-9e96-950d4603d0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040222 May 23 16:08:38.683368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1052/2213] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:38.780826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:08:38.781008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:08:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:08:39.205692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:39.206364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:39.212915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.213362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.214114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.214550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.277247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:39.284840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02dcb111-763f-4875-8265-9ae1343bd918 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082996 May 23 16:08:39.285406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1162/2214] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:39 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:39.311464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:39.312419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:39.319039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.319517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.323854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.323854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.349736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a7b9734-cc42-4815-b7b1-2ddf3c639e39 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042696 May 23 16:08:39.350295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1053/2215] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:39.696655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:39.697583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:39.703563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.704004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.704749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:39.705208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:39.740015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-574742c4-5600-4f45-9328-977f3db7846d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047202 May 23 16:08:39.740621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1163/2216] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:40.301648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:40.302262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:40.308435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.308889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.309646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.310083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.361364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:40.362209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:40.368610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.369058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.371009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.371009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.372980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:40.380641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf6ba893-7d32-4aed-8b15-44322a20e0f1 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082804 May 23 16:08:40.381209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1054/2217] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:40 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:40.401137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff45f50e-5022-489c-818f-f25cd880607f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043720 May 23 16:08:40.401758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1164/2218] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:40.752347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:40.753279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:40.759357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.759841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.760609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:40.761629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:40.789918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7fc4f636-a4b8-438d-a303-af958114efa4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041615 May 23 16:08:40.790476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1055/2219] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:41.398764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:41.399379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:41.406282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.406724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.407507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.407932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.412034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:41.412852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:41.413813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:41.421362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.429219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.429219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.429219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.455776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95421f87-8c41-4e54-bdbf-ae5821968987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046886 May 23 16:08:41.456345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1056/2220] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:41.476238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:41.484255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca89947f-d6c0-4d85-8c3f-d7144e48d592 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.089410 May 23 16:08:41.484807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1165/2221] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:41 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:41.801814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:41.802765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:41.808839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.809287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.810035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:41.810469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:41.838667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cacaa8a9-5ea3-42f4-be07-008c4bb57acd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040946 May 23 16:08:41.839293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1057/2222] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:42.466925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:42.467805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:42.474247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.474693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.475476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.475906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.504601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8cd5e36d-227c-4f91-bf3b-f37604399eb5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041751 May 23 16:08:42.505142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1166/2223] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:42.506831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:42.507492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:42.514195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.514649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.515722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.516157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.585776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:42.593339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-243fba68-fae6-4c1e-9b03-70e01863fef2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.090465 May 23 16:08:42.593901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1058/2224] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:42 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:42.850431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:42.851395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:42.857919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.858364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.859133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:42.859571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:42.888203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da626420-c2a7-4b76-a66b-6aa5f0e3e242 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041744 May 23 16:08:42.888741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1167/2225] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:43.516783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:43.517743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:43.524274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.524725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.525474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.525907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.553168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2317c70f-0c18-41df-ac86-fc9a33d42e1c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041056 May 23 16:08:43.553713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1059/2226] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:43.609844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:43.610455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:43.616878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.617318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.618057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.618489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.678977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:43.687120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aadb47d7-2bfb-4cc3-a5a3-6074d53c6314 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.081348 May 23 16:08:43.687668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1168/2227] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:43 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:43.899967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:43.901638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:43.907598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.908045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.908798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:43.909228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:43.938965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a1a151f2-206a-4caa-bec7-6c5bf5f5e775 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043026 May 23 16:08:43.939538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1060/2228] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:44.566401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:44.567321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:44.581670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.582129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.582896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.583366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.612102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1fc8c1a9-b0eb-44f8-863b-62e5e5240391 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051200 May 23 16:08:44.612724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1169/2229] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:44.704293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:44.704905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:44.711346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.711801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.712553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.713003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.775282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:44.783442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-87810c9d-0268-4501-81aa-b48939bae830 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083421 May 23 16:08:44.784006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1061/2230] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:44 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:44.950877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:44.951754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:44.958077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.958518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.959300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:44.959742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:44.987145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-028c6081-42fd-4239-bc78-efb402c82290 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040793 May 23 16:08:44.987681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1170/2231] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:45.623485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:45.624408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:45.630285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:45.630731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:45.631505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:45.631944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:45.647847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:45.660233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8eccf9cf-8c2f-4d70-84fc-bb8750f5f987 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040734 May 23 16:08:45.663348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1062/2232] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:45.800631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:45.801255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:45.807320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:45.807775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:45.808538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:45.808980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:45.868123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:45.875511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-22def54b-51f7-4583-a033-a3f7c298ee3d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.078733 May 23 16:08:45.876072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1171/2233] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:45 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:45.998404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:45.999352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:46.006284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.006734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.007529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.007960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.037018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c46331b-f7d8-41af-af85-a4bf47204193 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042594 May 23 16:08:46.037555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1063/2234] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:46.671810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:46.672666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:46.679216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.679673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.680432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.680878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.710453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ffbe4ac4-c9fe-4e7d-8f26-84c608b2ed4b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042707 May 23 16:08:46.710994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1172/2235] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:46.729292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:08:46.729463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:08:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:08:46.892218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:46.892838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:46.898915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.899409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.900181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:46.900626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:46.959970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:46.967397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eaddbb13-a2ab-40c9-aedb-14a268b6a5ec tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079203 May 23 16:08:46.967963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1064/2236] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:46 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:47.048803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:47.049733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:47.055763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.056215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:47.056972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.057411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:47.086307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bce6bd82-9b58-4025-ae47-292197fdab6c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041444 May 23 16:08:47.086852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1173/2237] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:47.722080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:47.722937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:47.729430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.729883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:47.730634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.731147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:47.758598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83927c31-33b8-45db-af4b-073aafae5f4d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040572 May 23 16:08:47.759149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1065/2238] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:47.984046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:47.984669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:47.991287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.991746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:47.992504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:47.992945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:48.052698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:48.060577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7058ad4-c99e-4233-8e72-895eb81c36f0 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080413 May 23 16:08:48.061144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1174/2239] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:47 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:48.097545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:48.098478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:48.104450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:48.104898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:48.105658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:48.106134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:48.134342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7521e8b-6bfd-461a-97c6-8747e9fbeac4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040692 May 23 16:08:48.134877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1066/2240] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:48.770201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:48.771187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:48.777145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:48.777590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:48.778865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:48.779321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:48.807974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4ec8058-7409-40a2-9b4f-3a90f7ce2f20 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041746 May 23 16:08:48.808516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1175/2241] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:49.077698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:49.078324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:49.084938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.087071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.087071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.087071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.147708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:49.148668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:49.148995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:49.155550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.155997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.156755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.157274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.160726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70a72640-d89a-4a08-a3d8-1a97dddfca92 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.087337 May 23 16:08:49.161565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1067/2242] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:49 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:49.185956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f1bfdce-a47c-4517-a6c3-6083a90b5179 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042216 May 23 16:08:49.186565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1176/2243] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:49.819605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:49.820581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:49.827173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.827634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.828394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:49.828834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:49.857062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dc2d61c3-9a5b-4cd4-a87e-fbaad9507de4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041500 May 23 16:08:49.857604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1068/2244] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:50.177109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:50.177785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:50.184805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.191123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.191123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.191123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.197626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:50.198594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:50.205894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.206354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.207181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.207623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.242778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ffb8e123-34b0-451e-a269-becfad650304 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049115 May 23 16:08:50.243381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1069/2245] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:50.252107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:50.259872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f672905c-5d5d-4374-bd66-5205614dae03 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.086689 May 23 16:08:50.260423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1177/2246] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:50 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:50.868134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:50.869047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:50.875518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.875962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.876709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:50.877138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:50.905899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-78e8cc15-0596-4340-b82a-4af9fe356c59 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042116 May 23 16:08:50.906429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1070/2247] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:51.253852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:51.254803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:51.261250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.261692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.262433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.262862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.277340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:51.277961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:51.284343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.284802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.285569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.286017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.291449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83f623e4-e9cb-4f98-8c72-42ce079a5bd0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041379 May 23 16:08:51.291999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1178/2248] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:51.345804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:51.353312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb7789eb-1716-4a8d-acdd-0142ca753192 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.079830 May 23 16:08:51.353873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1071/2249] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:51 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:51.916474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:51.917399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:51.923976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.924425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.925170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:51.925601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:51.953329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-688bd078-a2f4-4f59-8f53-ba0b7177716d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040551 May 23 16:08:51.953860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1179/2250] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:52.303426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:52.304364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:52.310840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.311334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:52.312100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.312540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:52.340901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-74a445f3-8623-4cb2-9e78-eb8b68609b19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041447 May 23 16:08:52.341956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1072/2251] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:52.370395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:52.371006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:52.377634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.378074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:52.378817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.379301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:52.439452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:52.446765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6fd36c19-9548-46c4-bc24-12794e7f0907 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.080276 May 23 16:08:52.447359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1180/2252] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:52 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:52.965019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:52.966666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:52.972620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.973070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:52.973835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:52.974271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:53.002890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-795629a5-ad48-4804-94d1-f3f478ac075d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041833 May 23 16:08:53.003466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1073/2253] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:08:53.353005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:53.353944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:53.359987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:53.360441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:53.361194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:53.361641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:53.385355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:53.390227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe4d0dde-8686-4c03-b533-01cd2bd434b7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041204 May 23 16:08:53.391029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1181/2254] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:53.463906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:53.464527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:53.471645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:53.472103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:53.472863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:53.473303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:53.535710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:53.543291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cebb9f49-26b6-4a06-939b-2ffc69f0126d tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083111 May 23 16:08:53.543860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1074/2255] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:53 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:54.014456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:54.015706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:54.022051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.022503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.023289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.023737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.052213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28f1e510-60e9-4db4-801e-3535a0524085 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041675 May 23 16:08:54.052759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1182/2256] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:54.416362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:54.417252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:54.427090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.427568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.428371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.428813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.461699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0fabb4c-18a8-4ed5-969e-647b9c3cc3fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049523 May 23 16:08:54.462256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1075/2257] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:54.560461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:54.561084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:54.567588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.568042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.568803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:54.569242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:54.631688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:54.639316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c8b1e48-0f36-4f18-95f3-c56f482c6a19 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082679 May 23 16:08:54.640773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1183/2258] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:54 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:55.064508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:55.065448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:55.071561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.072013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.072771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.073208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.101950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-008a9827-08bf-4c57-89a9-5af21c758e2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042193 May 23 16:08:55.102536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1076/2259] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:55.476029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:55.476892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:55.483064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.484060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.484832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.485302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.514108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7281443e-8278-4490-8be2-46ca5b896c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042107 May 23 16:08:55.514670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1184/2260] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:55.657119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:55.657878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:55.668226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.668675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.669463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:55.669898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:55.729848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:55.737841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6d2a652-e798-4435-916a-079a54953a0a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.084766 May 23 16:08:55.738398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1077/2261] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:55 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:56.205790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:56.206714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:56.213462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.214322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.215076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.216337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.247866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67d273f8-7c01-45cf-8ce2-13121f090c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.138376 May 23 16:08:56.248607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1185/2262] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:56.526376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:56.527337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:56.533899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.534353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.535134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.535682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.564307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e17b5cdc-9067-45cc-bcc6-81ea095d9f2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041890 May 23 16:08:56.564854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1078/2263] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:56.828973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:56.829600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:56.837823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.838293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.839077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:56.840881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:56.903528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:56.911010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b8097c7-2ee5-4b06-b22a-b4354313688a tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.159888 May 23 16:08:56.911619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1186/2264] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:56 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:08:57.258725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:57.259740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:57.261573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=90 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:57.266557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.267001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.267795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.268232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.271924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:57.298532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3d9f0367-a17a-4dda-a10e-d93ccf143906 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043746 May 23 16:08:57.299089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1079/2265] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:57.574964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:57.575933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:57.582860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.583352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.584121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.584563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.614353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f42126b-a41e-451f-a8ec-56975fe77c4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043352 May 23 16:08:57.614901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1187/2266] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:08:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:08:57.928593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:57.929130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:57.935868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.936316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.937104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:57.937538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:57.999311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:58.007185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-222a447f-23fa-411e-b1cc-205fc0ba7826 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.083101 May 23 16:08:58.007739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1080/2267] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:57 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:08:58.310246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:08:58.311231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:58.317904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:58.318357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:58.319126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:58.319593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:58.417496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e24654c0-67dc-46bf-8299-a05a95d3a5a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action" status: 200 len: 75904 microversion: 2.1 time: 0.110981 May 23 16:08:58.418139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1188/2268] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 16:08:58 2022] POST /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2/action => generated 75904 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 23 16:08:58.627835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:08:58.628693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:58.635436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:58.635903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:58.636660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:58.637091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:58.735434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25529783-feb6-48ca-b905-05a97e8dcb02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action" status: 200 len: 76307 microversion: 2.1 time: 0.111592 May 23 16:08:58.736214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1081/2269] 10.222.0.32 () {60 vars in 1277 bytes} [Mon May 23 16:08:58 2022] POST /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539/action => generated 76307 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:08:59.026039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:08:59.026745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:08:59.034356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:59.034808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:59.035588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:08:59.036027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:08:59.096689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:08:59.104703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cdb84b7b-6269-4849-a024-8c72082bfbbd tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082967 May 23 16:08:59.105253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1189/2270] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:08:59 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:00.122934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:00.123671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] [instance: 909fad86-0068-4016-bf26-7246f0b13e53] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:00.131401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:00.131847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:00.132599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:00.133034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:00.194025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6ed5088d3494139b93a7d2af4b48a12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '652580a82e9c4f3b84eda950e5613a48', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:00.201440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d60a83c3-843f-4d64-9b82-d8cdb43e2eaa tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53" status: 200 len: 1220 microversion: 2.1 time: 0.082756 May 23 16:09:00.201992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1082/2271] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:00 2022] GET /compute/v2.1/servers/909fad86-0068-4016-bf26-7246f0b13e53 => generated 1220 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:00.427455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34d71d82-7cf7-4443-92d9-6ea3561354e2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] Calling method '.version_select at 0x7f47f93280d0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:00.505775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34d71d82-7cf7-4443-92d9-6ea3561354e2 tempest-TestShelveInstance-8830984 tempest-TestShelveInstance-8830984-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1375701734" status: 202 len: 0 microversion: 2.1 time: 0.082036 May 23 16:09:00.506559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1190/2272] 10.222.0.32 () {58 vars in 1253 bytes} [Mon May 23 16:09:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1375701734 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 16:09:02.463706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:02.464866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:02.471949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.472659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.473613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.474212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.501977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:09:02.528136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:02.528983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:02.530570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:02.541898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.545538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.545840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.546147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.595115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:09:02.926455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6ad98736-c0b3-47b2-aadf-15be390be6b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 204 len: 0 microversion: 2.1 time: 0.466061 May 23 16:09:02.927217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1083/2273] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:09:02 2022] DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:09:02.933525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:02.936444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:02.944029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.944575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.945431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.946060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.965686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a57e1ba3-138a-4cae-94b0-7163457932e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 204 len: 0 microversion: 2.1 time: 0.441039 May 23 16:09:02.967283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1191/2274] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:09:02 2022] DELETE /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:09:02.976404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:02.977265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:02.985281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.985520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:09:02.986199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:02.987265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:02.988136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.020410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:09:03.299434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3480391a-e46e-4f29-ada4-c181eb919816 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 204 len: 0 microversion: 2.20 time: 0.370569 May 23 16:09:03.303126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1084/2275] 10.222.0.32 () {60 vars in 1279 bytes} [Mon May 23 16:09:02 2022] DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 16:09:03.306041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:03.306813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:03.313246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.313707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.314481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.314957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.331863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5ea491b-f478-4b0b-9ff1-4ef9a64dd0a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 204 len: 0 microversion: 2.1 time: 0.361013 May 23 16:09:03.333698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1192/2276] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:09:02 2022] DELETE /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:09:03.338011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:03.338718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:03.345069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.345634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.346480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.346990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.548929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.552241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.558766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c5f53219-7bc5-455e-b78d-da7a3776d68a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1730 microversion: 2.1 time: 0.257556 May 23 16:09:03.560218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1085/2277] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1730 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:03.560462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-397dd812-50a1-4a51-893b-0397555e6ad8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.226408 May 23 16:09:03.560951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1193/2278] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:03.565601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:03.566222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:03.572435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:03.572762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.572932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:03.573538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.574492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.575190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.584754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.585421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.586594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:03.587025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:03.796877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.800902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.807625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.809144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:03.809455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45037dd8-afc0-4b0d-b528-7b1d3e34d0fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.247331 May 23 16:09:03.810034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1194/2279] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:03.817901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02e92060-d50d-496c-bace-e43d2f0ce581 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.248757 May 23 16:09:03.817901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1086/2280] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:04.577482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:04.579474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:04.587131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.587728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.588600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.589198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.590302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:04.591534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:04.604077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.604538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.606076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.606076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.837263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:04.845632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:04.847908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f455f59e-848e-4c4f-8ee5-51cb121b2fa2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1730 microversion: 2.1 time: 0.275176 May 23 16:09:04.848553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1087/2281] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1730 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:04.857176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4381ce9e-7035-4ba9-8b83-98e9daf930a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.272174 May 23 16:09:04.857589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:04.858338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1195/2282] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:04.861053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:04.868434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:04.868434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:04.875401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.876082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.876948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.877549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.878655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.879416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.880800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:04.882531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:04.916180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:05.199941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:05.209411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-90086050-469a-4b70-abb1-e69f52700e3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.359628 May 23 16:09:05.209983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1088/2283] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:05.239463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:05.254856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:05.256711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:05.258124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-09e43406-df6f-47be-b68f-2d18ef6ad512 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.398344 May 23 16:09:05.271381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1196/2284] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:05.349769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-5e27bd85-0175-4b6b-997b-5ac4c3391045 req-69c7a8dd-63aa-4759-b2f1-a1472a3b9412 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4c5a0e23-b645-46dd-8924-37f041a76f7c", "name": "network-vif-deleted", "tag": "300c42fc-ece8-4ec8-987d-9dadf16db567"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:09:05.360294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5e27bd85-0175-4b6b-997b-5ac4c3391045 req-69c7a8dd-63aa-4759-b2f1-a1472a3b9412 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:05.361402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5e27bd85-0175-4b6b-997b-5ac4c3391045 req-69c7a8dd-63aa-4759-b2f1-a1472a3b9412 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:05.380281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-5e27bd85-0175-4b6b-997b-5ac4c3391045 req-69c7a8dd-63aa-4759-b2f1-a1472a3b9412 service nova] Creating event network-vif-deleted:300c42fc-ece8-4ec8-987d-9dadf16db567 for instance 4c5a0e23-b645-46dd-8924-37f041a76f7c on hv-cinder-83588 May 23 16:09:05.385996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-5e27bd85-0175-4b6b-997b-5ac4c3391045 req-69c7a8dd-63aa-4759-b2f1-a1472a3b9412 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101093 May 23 16:09:05.386609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1089/2285] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:09:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:09:05.875406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:05.877087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:05.884418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:05.885726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:05.886170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:05.886889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:05.888180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:05.888840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:05.897321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:05.897321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:05.897644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:05.898089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:05.989020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:05.999871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96342e1f-fb3b-4d4f-bea3-8b9eba28a6bb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.128372 May 23 16:09:06.006958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1197/2286] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:05 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:06.152809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:06.162148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10fe861c-7144-4a75-bad1-56c5e42dad4a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.284286 May 23 16:09:06.163705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1090/2287] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:06.225904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:06.226526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:06.233884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:06.234649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:06.235877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:06.236509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:06.277679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:06.278290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:06.284986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:06.285437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:06.286195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:06.286625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:06.795581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:06.803582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8dd5f9f-5e76-46c1-91aa-a91a2cb78cc2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.581499 May 23 16:09:06.804490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1198/2288] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:06 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:06.812704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:06.821132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:06.822644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:06.823845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5575ff01-c58f-4cd4-a668-7bc90689ebc0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.550028 May 23 16:09:06.824400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1091/2289] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:06 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:07.023962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:07.024627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:07.031562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.032222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.032997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.033444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.101095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:07.109347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-88a1dd12-d145-4677-8fbd-74fecfb349e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089304 May 23 16:09:07.110073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1199/2290] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:07.181861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:07.182464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:07.189689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.190146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.190928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.191393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.351628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:07.360896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60187fef-f4ae-4b9f-9e7a-5c085f9ff68a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.183432 May 23 16:09:07.361469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1092/2291] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:07.821282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:07.821842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:07.827901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.828377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.832315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.832315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.842337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:07.843195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:07.850298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.851119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.852090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:07.852532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:07.874341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:08.037601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.047758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f576dc40-c64f-495d-b0c2-8d9a7bdc031b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.229759 May 23 16:09:08.057401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.059574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1200/2292] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:08.064178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.065696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.066948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c0dc7b75-6b58-407f-9655-9397e8f16ce1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.228523 May 23 16:09:08.067548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1093/2293] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:08.127879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:08.128541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:08.136580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:08.137256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:08.138298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:08.138894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:08.243935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.253250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e2c99b1-0c97-4805-aa76-f7867671322c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.131115 May 23 16:09:08.253991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1201/2294] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:08.378465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:08.379078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:08.387770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:08.388216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:08.389262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:08.389699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:08.527380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:08.535617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7633480c-95af-4620-a3ec-feb4c558f10c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161654 May 23 16:09:08.536176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1094/2295] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:09.065700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:09.066323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:09.073008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.073502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.074263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.074698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.085230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:09.086063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:09.092044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.092496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.093247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.093679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.268056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.276358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-66c1227c-035b-4a2c-a933-65d004c2d9a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215245 May 23 16:09:09.276935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1202/2296] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:09.281902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.282166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:09.282781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:09.289149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.289669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.290493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.290651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.291066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.292197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.293367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fb5d7e65-0bc1-493f-83ea-27c81188b9db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212166 May 23 16:09:09.293927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1095/2297] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:09.354129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.362158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-165ea62e-2c7e-4a0c-a8d7-73bef605ec90 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084276 May 23 16:09:09.362727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1203/2298] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:09.553249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:09.553781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:09.560439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.560895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.561690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:09.562126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:09.709440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:09.717325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d48457c-7080-47e0-8108-dbcc673c74d8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.168486 May 23 16:09:09.717883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1096/2299] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:10.293924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:10.294544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:10.301104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.301560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.302327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.302771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.313041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:10.313661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:10.319655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.320109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.321069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.322162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.493423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.502197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e05bd6e8-bb08-411c-b598-831a39aabd61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212108 May 23 16:09:10.502826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1204/2300] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:10.504129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.512997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.514484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.516477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-94228bbd-99c6-4165-bf53-77f20b4135c5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.206793 May 23 16:09:10.517225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:10.517963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:10.524201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.525275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.527693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1097/2301] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:10.534199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.534860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.605686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.614485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a26f8c13-75c7-4b99-8a7b-d2bb815671ec tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.110585 May 23 16:09:10.615127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1205/2302] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:10.734675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:10.735326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:10.741918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.742372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.743156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:10.743595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:10.873451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:10.882090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a41da2a-750c-4ec2-8c79-8e40dc1b31ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151219 May 23 16:09:10.882665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1098/2303] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:10 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:11.520206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:11.520832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:11.527199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.527659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.528419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.528934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.534288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:11.534970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:11.541391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.542561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.542826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.543289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.739586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:11.744827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:11.747991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1df4964a-e94c-417b-a91e-3580df58639f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.232399 May 23 16:09:11.748561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1206/2304] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:11.753347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:11.754233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:11.755710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:11.757981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:11.759203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7cf3e70f-ad39-4530-8d58-3f4e05c5100a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.228918 May 23 16:09:11.759915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1099/2305] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:11.760277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.760795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.761564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.762080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.824801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:11.833230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2831be7-a9b1-4368-a595-3a2de982d551 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083730 May 23 16:09:11.833793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1207/2306] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:11.899627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:11.900346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:11.907032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.907522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:11.908289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:11.908734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.045800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:12.054262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9c102616-b649-4da1-801c-b061a7adeb95 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.158487 May 23 16:09:12.062053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1100/2307] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:12.769370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:12.770161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:12.777585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:12.778296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:12.778440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.778794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.779619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.780135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.785575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.786034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.786803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.787275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.968594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:12.976114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:12.978478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a71b921d-b319-4dea-bee3-3f415d207b61 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216614 May 23 16:09:12.979430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1208/2308] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:12.983725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:12.984329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:12.990847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.991330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.992081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:12.992507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:12.995054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:12.996691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:12.998141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-327660ed-ed5a-45e3-9208-ab3e8bfef79a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.224414 May 23 16:09:12.998777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1101/2309] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:13.056813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:13.065455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-52d7f57d-ec6f-40ee-b330-62df646f6d22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085125 May 23 16:09:13.066293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1209/2310] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:13.072104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:13.072725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:13.079004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:13.079486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:13.080245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:13.080842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:13.211903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:13.221045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdff9809-7c4c-4f43-adb3-29faf6628bbf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152784 May 23 16:09:13.221705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1102/2311] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:13.996746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:13.998011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:14.005067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.005836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.006856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.007622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.019456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:14.020169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:14.029550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.039561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.040437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.040890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.241519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.244679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.253301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b31df5d-ba12-4c01-85d4-61ad92dbfc0e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.260543 May 23 16:09:14.255608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.255878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1210/2312] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:13 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:14.263052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:14.264497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:14.265513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.267451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-640699e0-3aba-4104-a90a-f444e72a1464 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.253999 May 23 16:09:14.268541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1103/2313] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:14.274073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:14.274965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:14.277888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.278608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.280666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.281442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.281791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.286279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.287082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:14.287558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:14.371965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.384566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-255293e0-f2d9-4c9e-929a-9a97b0c7d13e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.129177 May 23 16:09:14.385773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1211/2314] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:14.459460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:14.472753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c6b9d3f2-f05a-4a75-9e42-ec3e25f10791 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.202842 May 23 16:09:14.474822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1104/2315] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:15.270580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:15.271311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:15.277832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.278354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.279252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.279781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.292274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:15.293018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:15.301938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.302604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.304854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.305606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.569627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:15.577717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de1643fd-0819-4456-a7fe-868536b3898e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.311073 May 23 16:09:15.578461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1212/2316] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:15.583265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:15.583991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:15.590004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.590589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.591508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.592053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.596297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:15.615600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:15.625004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:15.627160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:15.628440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d8d9f4f2-4ba2-4793-9bf5-2fb673bcc1a5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.340988 May 23 16:09:15.629306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1105/2317] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:15.634606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:15.635432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:15.642261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.642714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.643518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:15.644051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:15.694317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:15.702402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3971fda0-7c4c-48b8-bff6-087a8a8173fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.122613 May 23 16:09:15.703643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1213/2318] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:16.127061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:16.137130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0631c84d-d624-4e10-9c4c-ab8a8029ef82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.506088 May 23 16:09:16.137771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1106/2319] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:16.595497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:16.596235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:16.602922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:16.603467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:16.606176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:16.606176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:16.647903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:16.650071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:16.658723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:16.659200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:16.659973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:16.660403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:16.973499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:16.984172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dfa09665-0b0a-49ff-a91d-daa87938e75f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.392682 May 23 16:09:16.985455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1214/2320] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:16.990705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:16.991614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:16.998433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:16.998738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:16.999008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:17.000416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:17.000923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:17.008899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:17.010465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:17.011649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d6e65e35-0b99-4239-8ce7-bfacc063e23a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.368995 May 23 16:09:17.012215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1107/2321] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:17.109155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:17.117897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5e733fc-c5ed-48de-b66c-b4548f23c387 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.131134 May 23 16:09:17.118480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1215/2322] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:17.154940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:17.155725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:17.162848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:17.163574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:17.164600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:17.165199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:17.376387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:17.385159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af08a3ec-6e86-45f8-92f1-64e86583c6d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.234057 May 23 16:09:17.385730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1108/2323] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:18.003050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:18.003696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:18.013129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.013590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.014361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.014803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.035063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:18.035945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:18.044248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.047655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.047655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.047655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.250512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.258362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.267526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c7fa85ed-75ab-4d72-8a4a-1a788fbb6911 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.269389 May 23 16:09:18.271956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1216/2324] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:17 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:18.277048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:18.277369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.278173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:18.279594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.284615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5f05c71-3ad7-4bf3-9925-08576a326e21 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.253318 May 23 16:09:18.285393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1109/2325] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:18.285963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.286411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.287232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.287828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.361981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.371857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9de978cd-4f22-443e-b8b9-4e1128ff262a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.098531 May 23 16:09:18.372668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1217/2326] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:18.411380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:18.412067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:18.418538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.419270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.420268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:18.420870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:18.425678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:18.598814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:18.616302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-19b40040-6fbe-4121-9020-cc4d11656487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.215256 May 23 16:09:18.617068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1110/2327] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:19.284627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:19.285938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:19.293224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.293791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.294965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.295519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.305360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:19.306993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:19.323219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.323956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.324942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.326985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.610423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.629645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.631374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.636547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1e85175-2fc6-422c-897c-ecf52d3ec6bb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.336644 May 23 16:09:19.648933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.649875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1111/2328] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:19.655546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:19.656931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:19.662099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c32980b-58d4-428f-a515-ac4597c8033f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.381858 May 23 16:09:19.664147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.664838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.665894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.666573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.672920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1218/2329] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:19.681222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:19.687695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:19.697062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.697516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.698548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:19.699217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:19.767053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.777005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-103ef23d-8692-4262-badf-c3472a548acd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.125288 May 23 16:09:19.777845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1112/2330] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:19.912932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:19.921719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e3833c8-dd43-4c76-94c1-72461cef5b00 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.247276 May 23 16:09:19.929349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1219/2331] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:20.658342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:20.659829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:20.670375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:20.673472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:20.676436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:20.677105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:20.683491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:20.684919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:20.697589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:20.700147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:20.702222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:20.707149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:21.009855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.020520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.029518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.031053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.032954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2639301a-627b-4f55-8642-288279567f83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.379326 May 23 16:09:21.034394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1113/2332] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:21.037510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0eef3a22-8686-428a-8a53-94e61e889915 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.362135 May 23 16:09:21.039230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1220/2333] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:20 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:21.044396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:21.045156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:21.046157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:21.047182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:21.051371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:21.051828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:21.052607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:21.053049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:21.055850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:21.056486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:21.058154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:21.058603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:21.164450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.173157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4998c6f2-a4d7-422b-991d-79c9f6459706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.137708 May 23 16:09:21.176353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1114/2334] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:21.352143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:21.360986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da2769b4-0e7d-458a-b2cf-ee6297e007d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.320326 May 23 16:09:21.361503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1221/2335] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:22.060980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:22.062003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:22.062516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:22.063277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:22.073296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.074030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.075052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.075691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.076921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.078673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.079757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.080410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.446956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.450565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.460018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.462008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.463570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-499bc69c-6915-4272-abc6-83395fcc8ef3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.406543 May 23 16:09:22.464360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1222/2336] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:22.471064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:22.471976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:22.478316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.479790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.480777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.481447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.489275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8cc937e8-b561-4dc8-97f5-af1e6d9722bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.432482 May 23 16:09:22.498809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1115/2337] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:22.506012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:22.506638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:22.519932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.520844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.522517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:22.523422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:22.581285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.590085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-afcd72d4-ad5a-467c-b7b6-a247cfdb111e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.124370 May 23 16:09:22.592228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1223/2338] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:22.774541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:22.783205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf79d551-ad8c-42d2-80c1-7891861d861f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.291363 May 23 16:09:22.783777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1116/2339] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:23.481943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:23.483335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:23.490793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.491290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.492081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.492516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.509070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:23.511008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:23.518492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.519646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.520658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.521272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.778544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:23.788178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:23.790420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:23.790940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:23.792136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-008fee5c-2d66-409f-b312-1aa1365497ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.314207 May 23 16:09:23.804418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1224/2340] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:23.810602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ed690365-d550-48bd-b00e-42698a148941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.307805 May 23 16:09:23.811595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:23.813099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:23.819726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.820173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.820923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.821351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.823162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1117/2341] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:23.831328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:23.833294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:23.840499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.841169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.842116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:23.842549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:23.938771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:23.949291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2622612-2982-40be-83ca-3769f78382e7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.142033 May 23 16:09:23.950017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1225/2342] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:24.060667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:24.070524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a006774-f715-4d89-9412-1142b9d060b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.244169 May 23 16:09:24.071569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1118/2343] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:24.812353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:24.813509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:24.824279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:24.824990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:24.826310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:24.827046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:24.830251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:24.830999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:24.837965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:24.838776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:24.839947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:24.840988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:25.112094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.126648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.126648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.131955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.141417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2bd28aae-7003-4e7b-af2b-ebdd4d2902a8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.315765 May 23 16:09:25.142001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1119/2344] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:25.143372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70b27997-caba-4388-842f-10a09fd4d4d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.334476 May 23 16:09:25.144670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1226/2345] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:25.155244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:25.156798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:25.159792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:25.160676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:25.163973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:25.164428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:25.165228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:25.165663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:25.168075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:25.170115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:25.174539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:25.175038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:25.314557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.325772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1c0c0523-a671-4cbc-99f3-e95b3d668097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.179598 May 23 16:09:25.326804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1227/2346] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:25.445968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:25.455529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-45eb173c-b4bd-4ca8-9214-005cc47814d9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.307742 May 23 16:09:25.456114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1120/2347] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:25 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:25.811868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3939ea5c-1219-46ad-a0bd-2295de582486 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] Action: 'create', calling method: .version_select at 0x7f47f934de50>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1169274407", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKwL23h9VXygeaMx1GOdcyA2b9u4dtp3rUaMaMs3XOTilb98YQch1URDLdsmisJ3zJUSmBuSJxtPo4FaJdISpgZ0YqG4x+/9BNha+qWRKZ1YHv2wwpo9Qk/ssHol82PImA=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:09:25.826403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-3939ea5c-1219-46ad-a0bd-2295de582486 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] Getting quotas for project 01d73e969cd644d5a39fef629584bd12. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:09:25.830175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-3939ea5c-1219-46ad-a0bd-2295de582486 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] Getting quotas for user 789bc635e0dc4adea61f3adf29916258 and project 01d73e969cd644d5a39fef629584bd12. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:09:25.949697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3939ea5c-1219-46ad-a0bd-2295de582486 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.304601 May 23 16:09:25.950310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1228/2348] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 16:09:25 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:09:26.159185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:26.159816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:26.167314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.168591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.168591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.168939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.175246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:26.175862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:26.182255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.182705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.183523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.183956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.651854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.656878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.660865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eaf91d9a-8259-4004-9873-47f042daf48c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.505641 May 23 16:09:26.661433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1121/2349] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:26.665964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:26.666518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:26.667201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.668828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.670120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-42ecb33a-5c02-409a-a655-1f701338ebc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.505796 May 23 16:09:26.670962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1229/2350] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:26.673183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.673658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.674415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.674849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.676443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:26.677193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:26.679040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:26.685475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.686054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.686965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:26.687508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:26.743930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.752661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dfa0eddd-2505-45d8-93fb-696a5f11b32a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090197 May 23 16:09:26.757683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1122/2351] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:26.849831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:26.855219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:26.864427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bd9ab886-70c3-46cc-a866-2faf98dc72f8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.192302 May 23 16:09:26.865336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1230/2352] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:26 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:27.679849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:27.680483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:27.684349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:27.691825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.692009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:27.692278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.693158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.693465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:27.694078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.705870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.706442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.707366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.707904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.937006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:27.940096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:27.945290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-54d2f4dc-7e0f-4c54-a09a-a0b0138e00cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.270701 May 23 16:09:27.946058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1123/2353] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:27 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:27.949517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:27.950887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:27.951359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:27.951682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:27.952793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b7151e9-2ebc-44ed-9537-a7c11111d6aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.266571 May 23 16:09:27.953407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1231/2354] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:27.957938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.958174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:27.958667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.959050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:27.959605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.960253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.965564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.966229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:27.967213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:27.967860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:28.048345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:28.057599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4f6063c7-0919-4565-84ae-6f879ac7f777 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.110295 May 23 16:09:28.058184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1124/2355] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:28.141109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:28.154287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-42637858-7fe6-4b64-b9aa-ce76bef3fc01 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.199688 May 23 16:09:28.164106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1232/2356] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:28.963188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:28.963932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:28.971425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:28.972149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:28.973091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:28.973528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:28.974701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:28.975547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:28.985726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:28.985726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:28.985726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:28.985726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:29.179049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.183214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.188782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4e6a49d8-230f-4d55-8cf0-b91e1c44e636 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.229684 May 23 16:09:29.189362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1125/2357] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:28 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:29.194308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.195172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:29.195799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:29.195915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.198566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-54fe34f7-c459-4942-b8ef-fa7a9d54f4ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.227674 May 23 16:09:29.199173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1233/2358] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:29.204828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:29.205711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:29.210831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:29.211326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:29.211966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:29.212102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:29.212418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:29.212530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:29.213182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:29.213613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:29.304677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.308298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:29.314033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f46983f2-3cd3-45dc-b887-2397b3f98426 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.123184 May 23 16:09:29.314690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1126/2359] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:29.380808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:29.389615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17361f7c-15cd-42fa-b9f5-150a1a77958d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.189094 May 23 16:09:29.390188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1234/2360] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:30.206652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:30.207326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:30.214526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.215295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.216256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.216868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.218012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:30.218636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:30.225754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.226206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.227006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.227901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.423065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.426821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.431078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd8750f0-ba40-4583-ac03-fc95e4a28f9a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.229190 May 23 16:09:30.431663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1127/2361] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:30.440743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.440743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.440986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:30.441078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-426ed0b0-c7ae-470e-a50b-c5c29836914a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.224892 May 23 16:09:30.441165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:30.441255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1235/2362] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:30.443277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:30.443826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:30.444999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.445629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.446404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.446843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.449606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.450092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.450892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:30.451356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:30.519808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.527677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa03ccf8-7489-4194-9d9b-4f6762c5bfe3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093075 May 23 16:09:30.528238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1128/2363] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:30.601123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:30.609464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a9a957c-7880-445a-a5fc-f61874720ea2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169242 May 23 16:09:30.610023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1236/2364] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:30 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:31.448111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:31.448728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:31.456346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.456793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.457028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:31.457551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.457659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:31.457984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.464075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.464523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.465274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.465875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.654771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.658807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.663489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5effa7c8-82e4-4aae-8231-5375afbfe41c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219271 May 23 16:09:31.664063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1129/2365] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:31.667744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.669253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.670428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a54030a2-a23e-4463-8286-9b39db9b1207 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217874 May 23 16:09:31.672235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:31.672800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:31.674107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1237/2366] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:31.678947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:31.679434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.679601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:31.680029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.680903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.681519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.686231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.686693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.687737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:31.688205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:31.750935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.758907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0a878e9-91b7-441f-a1c7-006706bf2739 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091486 May 23 16:09:31.760962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1130/2367] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:31.837081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:31.844995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b146aa67-bec8-49cf-8fec-2bfc27eb92c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169826 May 23 16:09:31.845560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1238/2368] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:32.681787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:32.682391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:32.689435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.689835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:32.690277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.690446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:32.690647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.691087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.697131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.702095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.702095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.702095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.881746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:32.889611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5bad880b-da5b-4c8d-a780-0e5371ce649c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211888 May 23 16:09:32.890039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1131/2369] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:32.893059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:32.894781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:32.895425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:32.901281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.901768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.902535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.902963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.903239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:32.904745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:32.905994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b9b253d3-2a0d-4f6d-98ac-485dc6c3bbdc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220563 May 23 16:09:32.906547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1239/2370] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:32.917077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:32.917704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:32.924594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.925107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.925880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:32.926322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:32.969287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:32.977224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-50c51e32-3b50-436d-b028-af934b8d8792 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086218 May 23 16:09:32.977900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1132/2371] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:33.060765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:33.069248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4dd0e9a-d45b-4f72-a8b5-e7b06aa11038 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161764 May 23 16:09:33.069810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1240/2372] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:33.906864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:33.907546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:33.914361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:33.914824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:33.915664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:33.916110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:33.924686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:33.925294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:33.931922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:33.932491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:33.933254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:33.933690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:34.108288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.112155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.116691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa42aa33-566a-46d3-bb50-775187efa344 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214203 May 23 16:09:34.117353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1133/2373] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:34.121583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.122682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:34.123088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.123457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:34.125032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-513884a4-eb08-4d5e-9f5f-8759ed14ec59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.203537 May 23 16:09:34.128399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1241/2374] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:34.130236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:34.130782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:34.131683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:34.132223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:34.133218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:34.133840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:34.139858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:34.141190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:34.141954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:34.142392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:34.215435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.223344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ecf2ef1c-8e77-435d-86dc-7f504f50ba60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.104922 May 23 16:09:34.223910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1134/2375] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:34.285376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:34.293253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4c4b0730-fc3b-4d27-90e0-758634617959 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163859 May 23 16:09:34.293819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1242/2376] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:35.134443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:35.134992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:35.142660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.146380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.146508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:35.146932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.147032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:35.147164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.150733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.151220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.151993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.152433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.336526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.339347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.345756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.347299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.348378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f318e67a-c1a9-4f61-af46-2ff9809c6563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217938 May 23 16:09:35.348558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56de77d1-b91c-44f7-a8dd-d4ce06220e6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.209533 May 23 16:09:35.360927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1243/2377] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:35.360927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:35.360927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:35.361408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1135/2378] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:35.363321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:35.371338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.371487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:35.371584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.371584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.371584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.371828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.371828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.371828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:35.371828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:35.449679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.458086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ddda3545-cc44-4945-bdd6-9b0db1f15182 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107770 May 23 16:09:35.458654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1244/2379] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:35.521126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:35.529311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-221b4ce9-1c5f-4574-8816-73de8a9da0ce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169831 May 23 16:09:35.529872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1136/2380] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:35 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:36.366290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:36.367008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:36.368768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:36.369165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:36.369422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:36.373758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.374152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.374914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.375377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.375959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.376416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.377178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.377630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.557927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.566062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b7e7b572-71ca-482a-ac84-f1e58bb179c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.203786 May 23 16:09:36.568894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1245/2381] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:36.571234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:36.571860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:36.572864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.578387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.578907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.579634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.580068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.582026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.583562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.584721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a82d79b5-5f5a-4bf8-8aa0-b357b66f2c3f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220388 May 23 16:09:36.585269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1137/2382] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:36.589824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:36.590434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:36.597140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.597590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.598346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:36.598779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:36.644086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.652123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a9ccdbc4-7d69-4587-a03d-ee9f3fa0ccbc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084495 May 23 16:09:36.652698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1246/2383] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:36.737337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:36.745188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bc757e2e-0c17-4ccb-ba8e-9c32958bec5e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.158989 May 23 16:09:36.745746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1138/2384] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:37.582784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:37.583353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:37.589798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.590242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.590990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.591446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.603574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:37.606259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:37.610125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:37.611172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.611633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.612392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.612831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.774779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.783175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e6fc8e87-2488-47d7-9b28-e691d4a7523e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.204304 May 23 16:09:37.783762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1247/2385] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:37.788117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:37.788905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:37.794568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.795012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.795801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.796045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.796216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.805195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.806707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.807957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b3c06ba1-e87a-4e80-902d-7eadf8063363 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.208207 May 23 16:09:37.808514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1139/2386] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:37.812914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:37.813522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:37.819354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.819806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.820565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:37.821002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:37.859921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.867886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d843477-3218-4cfe-9018-b37cd43c34e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083147 May 23 16:09:37.868452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1248/2387] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:37.950751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:37.959150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-85199e5b-d188-4e90-bc2b-c7dd4ba0f157 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149692 May 23 16:09:37.959716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1140/2388] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:38.801457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:38.802326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:38.811016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:38.811854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:38.813818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:38.814611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:38.825592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:38.826233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:38.832633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:38.833087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:38.833848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:38.834280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:39.017337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.024104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.025332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6737ef8-1f23-4cdc-8a5c-6f2fb26207df tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.229211 May 23 16:09:39.025973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1249/2389] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:39.030420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:39.031040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:39.032830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.034341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.035579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06947239-9395-45b5-8a61-5524f10108c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.213763 May 23 16:09:39.036128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1141/2390] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:39.036792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:39.037249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:39.038017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:39.038456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:39.040651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:39.041258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:39.047584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:39.048158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:39.049041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:39.049556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:39.106545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.115019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28509610-1e96-4a86-8574-67f9f45d3a60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088157 May 23 16:09:39.115620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1250/2391] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:39.191399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:39.199257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a23b8f7-c413-4b15-9ddd-871ede82e97d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162108 May 23 16:09:39.199827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1142/2392] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:40.041725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:40.042340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:40.049574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.050022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.050812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.051275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.053660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:40.058434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:40.061002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.061454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.062732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.063183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.083207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:40.248038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.251685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.256496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f7534e1-c074-4f5d-a411-0f387c2a60dd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.218650 May 23 16:09:40.259132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1251/2393] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:40.260174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.261683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.262142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:40.262809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:40.262954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-817ee09d-3ad8-4835-ab02-1948984cc1f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.213140 May 23 16:09:40.266674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1143/2394] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:40.268867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:40.269503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:40.270019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.270530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.271385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.272186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.275216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.275675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.276430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:40.276866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:40.344234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.349512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03b233eb-c065-4ef7-a075-8ff8a71c3f96 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090063 May 23 16:09:40.350080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1252/2395] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:40.421787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:40.429755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a442ba9e-5e46-47eb-aba2-5d05753eed85 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.165156 May 23 16:09:40.430343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1144/2396] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:40 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:41.274242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:41.274879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:41.282213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:41.282656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.282797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.282907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:41.283491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.283932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.289504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.289961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.290724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.291185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.472002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.476152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.483983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-75e1a83c-e806-4d00-907f-155a2ab9cd2d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.209732 May 23 16:09:41.483983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1253/2397] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:41 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:41.485153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.485768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:41.486381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:41.486596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.487791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a1c88da-0b4d-4059-bdef-7b08b8de2fcc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.210172 May 23 16:09:41.498580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1145/2398] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:41.498759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.498868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:41.498999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.499124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:41.499223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.499223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.499995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.500484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.501194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:41.501792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:41.588194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.597314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1860099e-003b-47cf-839f-8c18ffb86abd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.115020 May 23 16:09:41.598004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1254/2399] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:41.654869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:41.662869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-56fd404c-939f-4505-b093-925a7201ad4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173554 May 23 16:09:41.663464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1146/2400] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:42.497402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:42.498015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:42.504511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.504955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.505515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:42.505914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.506123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.506239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:42.512704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.513156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.513912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.514346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.709557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.716356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.719280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9784504a-f435-4a7d-adfd-954881e2ffb2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.224119 May 23 16:09:42.719280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1255/2401] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:42.723447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:42.724069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:42.725538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.727042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.728253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-80e315fc-eac8-4421-87c8-7c166f8f7dc3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.226602 May 23 16:09:42.728824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1147/2402] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:42.729828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.730387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.731159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.731641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.733123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:42.733803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:42.739619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.740157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.740831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:42.741265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:42.800544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.809133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d8eb005c-fedc-495a-b7b8-83e534716b99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088621 May 23 16:09:42.809133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1256/2403] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:42.882190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:42.891338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ac9271cc-750f-4a29-abb6-0d9b5960c076 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160704 May 23 16:09:42.891913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1148/2404] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:43.736241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:43.736880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:43.743471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.743914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.744669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.745103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.745721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:43.746386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:43.753227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.753670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.754422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.754854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.945661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:43.949577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:43.954219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ab80b2ac-74b4-4e31-ae36-60ad06c0147d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.222456 May 23 16:09:43.954785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1257/2405] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:43.958574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:43.959251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:43.959857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:43.960248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:43.961435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7e1814e-e187-429e-83fb-5bc2bbe2156a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219554 May 23 16:09:43.962127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1149/2406] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:43.965593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.966037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.966789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.967251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.970889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:43.972180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:43.978016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.978538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:43.979391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:43.979913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:44.038001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:44.045866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-159c0e37-30eb-4733-bf2d-3e6c5a6a95b4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090109 May 23 16:09:44.046435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1258/2407] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:44.121678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:44.129613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5394ced7-a1f2-44d6-9cec-bc87a4e6fe90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162849 May 23 16:09:44.130179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1150/2408] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:44.972304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:44.972919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:44.979503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:44.979935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:44.980337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:44.980552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:44.980714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:44.981154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:44.986891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:44.987376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:44.988135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:44.988570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:45.171922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.175906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.179900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd02b836-ec61-4793-8764-4d2161a9a0d5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212043 May 23 16:09:45.180471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1259/2409] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:45.184215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.188886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:45.188886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:45.189103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.189103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4150fa63-9849-4331-8d17-512919e28d59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.210800 May 23 16:09:45.189103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1151/2410] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:45.192624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:45.193318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:45.199383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:45.199814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:45.199955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:45.200274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:45.200793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:45.201020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:45.201259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:45.201451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:45.289920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.297910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ac700d67-b569-4044-8306-06e2cb76634e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.116452 May 23 16:09:45.298478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1260/2411] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:45.353135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:45.362232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b328d691-caad-4c40-8551-37c4dcab0008 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173582 May 23 16:09:45.363116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1152/2412] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:45 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:46.197513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:46.198129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:46.205469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.205911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.206020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:46.206525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:46.206779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.207253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.213035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.213489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.214249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.214688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.401233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.404551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.409253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-53628179-408b-4709-9ab3-7be32bf812fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215675 May 23 16:09:46.409833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1261/2413] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:46 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:46.413261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.414142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:46.414745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:46.414864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.415974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d26296ca-8033-4a6e-9823-95282db57014 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214600 May 23 16:09:46.420587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.421035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.421810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.422238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.426175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1153/2414] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:46.432976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:46.433603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:46.440265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.440717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.441995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:46.441995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:46.489062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.497158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e1395479-2772-4c84-9abb-f953e38f75a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086355 May 23 16:09:46.497732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1262/2415] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:46.580755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:46.588623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa373914-0b57-4d4d-9804-2cffcaa0e632 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159851 May 23 16:09:46.589187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1154/2416] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:47.426378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:47.426957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:47.433660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.434001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:47.434396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.434620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:47.434869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.435345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.441541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.441991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.442749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.443209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.623896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.628839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.632103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-07e7c370-8642-4437-ab88-fe150c85a941 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210183 May 23 16:09:47.632689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1263/2417] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:47.637139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:47.637750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:47.638131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.639808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.641334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e7124b2a-365f-4011-a035-c20bda24a1e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.210970 May 23 16:09:47.642205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1155/2418] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:09:47.644551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.645012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.645800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.646266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.646590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:47.647234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:47.653063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.653519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.653883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:47.654475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:47.654705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:47.733710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.742229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6851a539-e5c8-4633-bd53-f101dae8f5b0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.108595 May 23 16:09:47.742791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1264/2419] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:47.799319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:47.807123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b9023df-c9ae-4726-9657-c11cb0911d40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.164067 May 23 16:09:47.807684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1156/2420] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:48.649573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:48.650189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:48.656715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.657160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.657906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.658340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.659556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:48.660251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:48.666615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.667070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.667861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.668303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.856504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:48.860658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:48.864481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-085f2ff4-594b-4643-8171-302675d63112 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.218710 May 23 16:09:48.865056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1265/2421] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:48 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:48.870155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:48.871044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:48.871716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:48.872133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:48.873089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0f02f7e-7f4e-4cf9-aa3e-10934a2b12c2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217350 May 23 16:09:48.875367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1157/2422] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:48.878771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.879017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:48.879229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.879653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:48.879982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.880412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.885413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.885867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.886624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:48.887089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:48.971081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:48.978868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-049341ec-f3f8-4f41-8b0f-72d92ae6cf8f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.112869 May 23 16:09:48.999791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1266/2423] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:49.038417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:49.046535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-46581dac-3369-4a78-8136-88a796d61b9c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.171496 May 23 16:09:49.047127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1158/2424] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:49.882006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:49.882621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:49.889130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:49.889574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:49.890319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:49.890750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:49.892554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:49.893164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:49.899555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:49.900003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:49.900759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:49.901195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:50.086850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.089786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.094918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c21e6e6d-f71e-40ac-9fc8-f1a7085216a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216862 May 23 16:09:50.095523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1267/2425] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:50.098215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.099752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.099948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:50.100559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:50.100908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-29648375-0336-4a3b-a273-c50e3e0d9c1b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212994 May 23 16:09:50.107144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1159/2426] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:50.107144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:50.107367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:50.107455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:50.107552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:50.107651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:50.108049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:50.112877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:50.113326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:50.114081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:50.114512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:50.118434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:50.180271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.189123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a926d8bc-29f4-4143-92e8-930259a9acda tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092041 May 23 16:09:50.189799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1268/2427] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:50.267644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:50.275489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0e89e898-1bc0-433c-94b0-dd6df8ea334c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.172846 May 23 16:09:50.276052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1160/2428] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:51.112341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:51.112957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:51.119527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:51.119983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.120096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.120192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:51.120844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.121277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.127200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.127663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.128451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.128880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.322594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.325836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.331306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.332906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.333855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1233f8f3-745e-4870-a2a9-76c280c229e4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.225861 May 23 16:09:51.334239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c7286179-ce7d-4219-9f4e-0245db99b5ad tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.218479 May 23 16:09:51.334425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1269/2429] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:51.345845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1161/2430] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:51.346044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:51.346167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:51.346271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:51.346378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:51.346492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.346492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.346669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.347176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.347348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.347614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.348118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:51.348621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:51.669470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.677650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-008236ff-9438-4169-a833-97560f80148e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.342270 May 23 16:09:51.678265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1270/2431] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:51.735630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:51.743566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2abc9437-78e5-4fe2-83d4-33759c95b2ee tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.407689 May 23 16:09:51.744135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1162/2432] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:51 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:52.350906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:52.351542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:52.352994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:52.353616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:52.358001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.358449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.359224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.359659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.360562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.361015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.361776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.362216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.545849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.550216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.554432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03d8fadf-2823-4932-a06c-12a88ad29651 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.207376 May 23 16:09:52.555233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1271/2433] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:52.558993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.563298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.563298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-47bdcd38-6094-409c-82c0-ac56561b1c7d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212562 May 23 16:09:52.563298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1163/2434] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:52.694568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:52.695206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:52.701187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.701614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.702370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.702805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.759693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:52.760310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:52.764028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.766765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.767242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.768009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:52.768446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:52.772228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce5373f7-33db-40ce-887b-7094ba149dc1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081620 May 23 16:09:52.772797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1272/2435] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:52.901646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:52.910041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e8030e66-c54b-4871-8736-bf9cb2424c4c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154268 May 23 16:09:52.910606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1164/2436] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:53.572368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:53.572916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:53.579616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.580081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.580838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.581127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:53.581576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.581750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:53.588353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.588807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.589579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.590017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.767635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:53.775626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e87576eb-cb2f-47ea-819d-b2840aff0514 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.207485 May 23 16:09:53.776457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1273/2437] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:53.778359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:53.787898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:53.788909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:53.789426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:53.789557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:53.790585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60a3e9a4-2c46-4e72-b37b-61a6d0654e7a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.213367 May 23 16:09:53.791154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1165/2438] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:53.795430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.795882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.796638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.797078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.856928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:53.864736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c5a9927b-e7a8-41ee-95ef-2ea0f8b6abe6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079646 May 23 16:09:53.865304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1274/2439] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:53.927067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:53.927714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:53.934421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.934865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:53.935658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:53.936094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:54.060394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:54.068423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa460707-f18d-47cc-8ff8-9c2a1fc324b8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145308 May 23 16:09:54.068993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1166/2440] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:54.793350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:54.793959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:54.800933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:54.801373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:54.802122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:54.802551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:54.808875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:54.809401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:54.816599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:54.817056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:54.817823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:54.818258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:55.009393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.016997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.017917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fcdf4889-81c2-4bf0-bbb6-9301403b871e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.228367 May 23 16:09:55.018498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1275/2441] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:54 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:55.023011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:55.023654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:55.025503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.027002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.028190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f0dfd20d-ba8c-4ff0-9359-e690e4499519 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.223486 May 23 16:09:55.028734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1167/2442] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:55.029526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:55.030043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:55.030884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:55.031349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:55.085556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:55.086080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:55.092251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:55.092599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.092748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:55.093474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:55.093908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:55.101174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9c3620b8-a319-487b-b52e-3401b824d339 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081728 May 23 16:09:55.101745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1276/2443] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:55.226208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:55.235885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4976282e-c3e0-4c09-a83a-78e5bfd8c1af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.153538 May 23 16:09:55.236756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1168/2444] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:09:56.035597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:56.036238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:56.042629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.043084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.043861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.044292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.046743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:56.047895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:56.054308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.054756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.055543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.055979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.237570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.241630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.245825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9ca8b5f-d037-4b63-8b20-e9c150877424 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214755 May 23 16:09:56.250424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1277/2445] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:56.252020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:56.252630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:56.259946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.260394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.260653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.262877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.262877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.263118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.263966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0afa7af-3e24-49ad-8f6c-281293b0a4ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.221145 May 23 16:09:56.264729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1169/2446] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:56.269494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:56.270134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:56.276482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.283589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.283589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:56.283589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:56.328753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.337103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20389dc8-b03c-4023-ae45-71c150dd46a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089328 May 23 16:09:56.337687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1278/2447] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:56.420391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:56.428938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b1ba50f5-40ad-402d-8eea-7b14e58d21e5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163269 May 23 16:09:56.429504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1170/2448] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:56 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:57.269636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:57.270257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:57.277034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.277483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.278241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.278702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.286397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:57.287022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:57.295574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.296035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.296834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.297278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.473750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.480449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.481761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f176cdb-a8ef-4e01-bd47-70b4bce5ace5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216757 May 23 16:09:57.482499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1279/2449] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:57.487607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:57.488152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:57.489353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.490797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:57.490932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.492020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c0d0f0c7-6a7a-4124-8073-b136c0fca1df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.209546 May 23 16:09:57.492570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1171/2450] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:57.494779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.495258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.496045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.496473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.496958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:57.497565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:57.505279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.505926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.510626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:57.510626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:57.580866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.584311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:57.589130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-253f1c6f-8f38-4004-9ec6-4272aa0cbe6c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.105669 May 23 16:09:57.589709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1280/2451] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:57.655710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:57.666278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f7843c75-ed53-4d4c-a0b1-19e9590b1814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.172763 May 23 16:09:57.666847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1172/2452] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:58.499991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:58.500611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:58.507480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.507930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.508688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.509125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.510663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:58.511332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:58.513829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:58.519461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.519914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.520727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.521169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.698886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.707462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-369a1d4a-9646-4c99-b31d-256c2a047092 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211645 May 23 16:09:58.708027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1281/2453] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:58.712614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:58.713223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:58.713572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.719137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.719590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.720347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.720785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.722504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.724100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.725336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9c6e5de7-21cc-432b-a6e5-ea3855e00991 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.218562 May 23 16:09:58.725983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1173/2454] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:58.730375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:58.731049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:58.736878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.737335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.738088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:58.738520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:58.784078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.792075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-044c1bc9-ce23-4417-af7d-f99d3e7de852 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083072 May 23 16:09:58.792652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1282/2455] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:58.874756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:58.882986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-867160ea-eaac-48c1-95ce-810b3b91cdd7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.156015 May 23 16:09:58.883568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1174/2456] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:59.725086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:59.725666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:59.732041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.732492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.733246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.733680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.743294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:59.743910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:59.750838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.751326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.752096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.752540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.940001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:59.947477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:59.947820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc020ec7-9353-4f26-a088-c6180b62cf22 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.226698 May 23 16:09:59.948396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1283/2457] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:09:59.952762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:59.953391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:59.956616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:59.958118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:09:59.959391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bdca8edb-bed0-4c27-b453-1fc664057f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219882 May 23 16:09:59.959562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.959947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1175/2458] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:09:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:09:59.960081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.960778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.961208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.964303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:09:59.965438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:09:59.971646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.972099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:09:59.972996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:09:59.973436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:00.030181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:00.038136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bac83a8c-5945-4405-8985-e970d2fafc72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088807 May 23 16:10:00.038701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1284/2459] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:00.119527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:00.124164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:00.130309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62116b11-67f4-4c78-86e9-7a74fe3de6c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169383 May 23 16:10:00.131191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1176/2460] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:09:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:00.965597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:00.966205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:00.973091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:00.973543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:00.974299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:00.974730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:00.980973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:00.982454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:00.993939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:00.994470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:00.995414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:00.995947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:01.179722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.187823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a2f33d3-d6b3-4ca8-8e28-0ebb87d7578a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.226172 May 23 16:10:01.189173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1285/2461] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:00 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:01.193934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:01.194546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:01.196115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.200658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:01.201112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:01.201910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:01.202393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:01.205066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.206560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.207761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5929a2b2-5a55-41d1-8e43-1f394a463db9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.233318 May 23 16:10:01.208310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1177/2462] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:01.212623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:01.213223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:01.219165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:01.220116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:01.220315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:01.220749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:01.270368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.279870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76944aaf-fed3-4949-9949-11b54cf3504a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089510 May 23 16:10:01.280673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1286/2463] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:01.368040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:01.376593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d49be4bb-58c8-430e-a216-47acd8f384c6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.167350 May 23 16:10:01.377163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1178/2464] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:02.205924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:02.206543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:02.213091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.213539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.214292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.214729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.226361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:02.226998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:02.233744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.234162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.234937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.235401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.419375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.424054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.428099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-42051977-9b6a-467f-b876-e0b5ce44545e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.226503 May 23 16:10:02.428674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1287/2465] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:02 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:02.433400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:02.434015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:02.434745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.436406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.437672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89284a08-0f4d-4e49-84b7-4f5b4737871a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.215169 May 23 16:10:02.438403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1179/2466] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:02.440064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.440508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.441264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.441695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.443417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:02.444200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:02.450331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.450791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.451616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:02.452057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:02.513048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.521442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f5d10bba-4db4-41a4-8ac9-eb98755b29b2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091822 May 23 16:10:02.523983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1288/2467] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:02.601158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:02.609181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84836359-0214-4386-929c-05ad5e847a6f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169682 May 23 16:10:02.609801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1180/2468] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:02 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:03.445674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:03.446295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:03.452927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.453380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.454769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.455233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.456328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:03.456965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:03.463560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.464136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.464909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.465406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.652595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.655825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.660792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff70c97a-9e1a-494c-99f2-d358ac2fd0c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219000 May 23 16:10:03.661660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1289/2469] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:03.664352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.665785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:03.665913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.666387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:03.667241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5ba3d6d9-0e55-4ad5-b151-21c12c40a1ef tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.215461 May 23 16:10:03.671182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1181/2470] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:03.672746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.673203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.673331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:03.673888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:03.674104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.674373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.679937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.680392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.681414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:03.681888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:03.757545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.765348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d14113f7-6eca-4ced-a96d-de5fd8fa36d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.102972 May 23 16:10:03.766009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1290/2471] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:03.824561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:03.832339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-71423c1b-776a-4ef9-ac76-464c737d0c1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163091 May 23 16:10:03.832887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1182/2472] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:04.624269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:10:04.624867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:10:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:10:04.678070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:04.678649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:04.685184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.685533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:04.685659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.686183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:04.686357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.686785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.692633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.693086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.693841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.694274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.875433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:04.881987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:04.883879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:04.885387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:04.886554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0794bdf4-21ac-4819-88ec-1b1a5cbdc2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.204820 May 23 16:10:04.887126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1183/2473] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:04.889961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c5d4538d-fb29-4327-b49d-c30fa67cf0ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215709 May 23 16:10:04.890641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1291/2474] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:04.895253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:04.895861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:04.898632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:04.899317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:04.901826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.902272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.903018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.903479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.905279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.905736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.906499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:04.907493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:04.993972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:05.002361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-755fac7f-b38f-466b-87d9-c89fb3325809 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.114042 May 23 16:10:05.005577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1184/2475] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:05.058583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:05.066417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c2c1e9cb-235f-4d01-958b-0ea94ee209a5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.175001 May 23 16:10:05.066971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1292/2476] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:05.905115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:05.905732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:05.907543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:05.908203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:05.913152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:05.913609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:05.914348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:05.914479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:05.914791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:05.914915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:05.915573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:05.916006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:06.104217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.108585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.112808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b66e2a6c-5260-410c-8cf1-e48c3465d888 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.209109 May 23 16:10:06.113382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1293/2477] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:06.117601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.117777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:06.118392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:06.119087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.120271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77ca9131-a4d1-4e52-8ecd-caa3cb36d053 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219116 May 23 16:10:06.124174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:06.124623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:06.125376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:06.125813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:06.127578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1185/2478] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:06.132181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:06.132791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:06.138631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:06.139166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:06.139925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:06.140358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:06.198591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.206982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-356e20ff-ab57-4242-8883-ef7fc6a06d2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092653 May 23 16:10:06.207590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1294/2479] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:06.280255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:06.288804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03e036cc-8686-487f-afcc-286b75aa46d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160176 May 23 16:10:06.290583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1186/2480] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:07.130284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:07.130933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:07.137534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.137694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:07.138095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.138215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:07.138729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.139195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.145407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.145849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.146628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.147078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.325633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.334027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f9b406b7-cbc5-49bf-a878-4c71f13e186f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208125 May 23 16:10:07.334595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1295/2481] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:07.339193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:07.339803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:07.339930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.346317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.346760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.347539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.347970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.348821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.350343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.351565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bfdc896b-b3f9-4ebd-8fb8-38d74e7e8ff7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217846 May 23 16:10:07.352121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1187/2482] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:07.356521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:07.357135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:07.362876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.363369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.364138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:07.364579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:07.415480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.423516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78841f43-da88-4b56-8ae1-71ff544b821d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087905 May 23 16:10:07.424118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1296/2483] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:07.506515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:07.514269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0480b3b5-52f7-4e94-8230-8b8b9f2eb31b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161218 May 23 16:10:07.523791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1188/2484] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:08.351509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:08.352131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:08.358685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.359165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.359925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.360363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.370216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:08.370833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:08.377831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.378281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.379051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.379513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.379671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:08.560847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.564009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.568997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbd004d3-8b0e-4ea9-a324-6b77f7f85563 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.221290 May 23 16:10:08.569669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1297/2485] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:08 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:08.572383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.574167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.574753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:08.575373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ecf3ceeb-0cac-4ae4-bdcd-3f63dc40a2a3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.209262 May 23 16:10:08.575510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:08.575913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1189/2486] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:08.580536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:08.585771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.585915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:08.586025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.586025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.586025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.587118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.587609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.588400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:08.588837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:08.651772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.660270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fafd097b-4d3f-4f41-beb8-9b518edd4492 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089567 May 23 16:10:08.660852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1298/2487] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:08.735912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:08.743775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1e1eb30c-980c-4b8f-a72a-8ed38ff9ebfd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.166709 May 23 16:10:08.744328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1190/2488] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:09.588312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:09.588858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:09.592692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:09.593427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:09.595428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.595883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.596648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.597086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.599923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.600369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.601118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.601546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.787841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.791188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.800677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.802175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.803358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b81abd63-7da5-4e9f-92f7-68cb40015ee5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214560 May 23 16:10:09.815263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1191/2489] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:09.815479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4df7de74-2dab-4099-83a3-8b5566e6a0b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220964 May 23 16:10:09.815479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1299/2490] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:09.815669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:09.815669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:09.815840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:09.815840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:09.817518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.817970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.818732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.819214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.823827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.823827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.823827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:09.823827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:09.890676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.899076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0394b178-6d24-4194-a5e5-1b1f7fc01845 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.094217 May 23 16:10:09.899728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1192/2491] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:09.972025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:09.980879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8464e5a9-e2ff-4857-a296-5b58b30569b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173442 May 23 16:10:09.981490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1300/2492] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:10.822689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:10.823309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:10.823549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:10.824181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:10.830383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:10.830828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:10.831150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:10.831608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:10.831721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:10.832036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:10.832369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:10.832811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:10.849632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:11.024877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.028061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.033848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.035433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.036122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-25dc574b-bbaa-4eaf-ad34-fcb55376858e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216678 May 23 16:10:11.036680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3f1b14d4-722f-400f-9218-c1306f36ab38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217923 May 23 16:10:11.036845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1301/2493] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:11.045947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1193/2494] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:11.046156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:11.046257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:11.046358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:11.046447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:11.048708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:11.048911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:11.049160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:11.049343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:11.049930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:11.050110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:11.050815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:11.050978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:11.136665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.144934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae6bf227-947c-43c4-9c0e-e6ac805bafbd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107290 May 23 16:10:11.145503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1302/2495] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:11.200364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:11.208251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e9b16d5c-ad6c-48fd-8618-f94f2d8a5d3e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169828 May 23 16:10:11.208809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1194/2496] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:12.053794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:12.054419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:12.061141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.061595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.062359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.062810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.063446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:12.064671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:12.071279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.071736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.072510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.072982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.253214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.259448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.261257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a82da2b6-80cd-4085-8160-7afbb4a41211 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211293 May 23 16:10:12.261832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1303/2497] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:12.266569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:12.267241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:12.268462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.270375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.271582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-18e254e2-363d-416e-839c-32a8cf245da9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212063 May 23 16:10:12.272138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1195/2498] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:12.272961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.273410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.274173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.274610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.276640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:12.277262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:12.283794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.284244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.285015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:12.285448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:12.347233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.355038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d962cf4b-5aae-483f-91c8-62d79ca257e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092259 May 23 16:10:12.355634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1304/2499] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:12.427944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:12.435869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4b9aba6f-ba28-4472-a3b9-8d849aa84cf7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162793 May 23 16:10:12.436429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1196/2500] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:12 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:13.278595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:13.279366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:13.286223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.286670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.287485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.287926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.289622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:13.290266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:13.296881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.297336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.298097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.298531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.490941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.495199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.499072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0664cbb2-8623-4ed0-9a0f-71910d4a79fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.224362 May 23 16:10:13.499656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1305/2501] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:13 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:13.504384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.504804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:13.505409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:13.505896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.507116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa1bbaa3-075a-45af-9c15-0a6ff89d1a99 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.221328 May 23 16:10:13.507918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1197/2502] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:13.511544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.511990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.512745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.512857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:13.513173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.513358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:13.520359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.520751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.521576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:13.522023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:13.584607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.592764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-23f4939b-5e97-4613-9d01-e70facf979c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092139 May 23 16:10:13.593329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1306/2503] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:13.673519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:13.682167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bd7d08d3-19eb-476f-a26a-1602d0b26e98 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173295 May 23 16:10:13.682729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1198/2504] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:14.527512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:14.528120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:14.530094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:14.530721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:14.534678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.535150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.542050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.542050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.542241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.542241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.542241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.542241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.725463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.729753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.734370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.735930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.737676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-abe87e0d-2661-47ca-97aa-f63a28673e0b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214238 May 23 16:10:14.738574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1307/2505] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:14.741387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-470460c4-b86d-4bdd-85e4-0c3b38ed3872 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215406 May 23 16:10:14.748686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1199/2506] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:14 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:14.748866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:14.748866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:14.749044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:14.749044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:14.749733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.750190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.750957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.751511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.753650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.754092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.754838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:14.755302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:14.834769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.842602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-973c27c7-25a0-48d9-8b6a-3b318ee5a04f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.103219 May 23 16:10:14.843187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1308/2507] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:14.901001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:14.908916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b7e0c425-54ba-4e73-9d1b-beae5d9e820b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.165967 May 23 16:10:14.909477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1200/2508] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:15.756319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:15.756936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:15.757782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:15.758401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:15.763530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.763981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.764732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.764853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.765169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.765284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.765911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.766607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.962510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:15.966671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:15.970951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-921604aa-bdd7-4522-8199-bc966e8d0d08 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216842 May 23 16:10:15.971555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1201/2509] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:15.975780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:15.976027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:15.976587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:15.977571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:15.978750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9ce09ef4-1477-4ca5-8871-4f96e8fe40b5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.226406 May 23 16:10:15.981183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1309/2510] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:15.982666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.983218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.984620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.985072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.990069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:15.992174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:15.997010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.997462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:15.998377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:15.998813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:16.056846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:16.065633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-330cd1b9-8402-43a3-99d9-8e9a53cf449f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092788 May 23 16:10:16.066482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1202/2511] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:16.143311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:16.150918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf71fcbb-a0ba-4223-a87c-3fbe8ed800ba tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.168736 May 23 16:10:16.151531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1310/2512] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:16.987686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:16.988297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:16.995168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:16.996848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:16.996848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:16.996848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:16.997167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:16.997167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:17.004192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.004643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.005411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.005846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.429087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.437138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53d5a789-3828-4be1-a481-09a124cac289 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.453260 May 23 16:10:17.437718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1203/2513] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:17.438819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.442223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:17.443373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:17.448502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.449233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.449685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.450023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.450469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.450885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.451230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1250019-46dd-4923-9e4e-2f04ebdf20f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.458331 May 23 16:10:17.452008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1311/2514] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:17.458789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:17.459447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:17.465398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.465846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.466613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:17.467087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:17.513661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.522626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-79fc021d-7eab-42d8-93cb-6df09064e4a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083807 May 23 16:10:17.523231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1204/2515] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:17.604816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:17.612807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-812f1bb9-195d-4296-983e-21f448506886 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159929 May 23 16:10:17.613368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1312/2516] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:18.454330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:18.454859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:18.461883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.462333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.463090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.463564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.469737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:18.470447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:18.478666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.479241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.480000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.480434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.646452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.656544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84daa956-52a1-452b-95ed-d6ce8f764792 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.206130 May 23 16:10:18.657147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1205/2517] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:18 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:18.661632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:18.662243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:18.669819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.670256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.670736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.671709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.672056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.678796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.680347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.681524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e70f9e52-53e7-4672-8a57-c294451b56fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.215772 May 23 16:10:18.682091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1313/2518] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:18.686514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:18.687143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:18.692980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.693430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.694793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:18.695265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:18.699265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:18.735567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.744210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-98ec372a-6f20-4c3b-b715-bfbcd76525f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086071 May 23 16:10:18.744783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1206/2519] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:18.829808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:18.837717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-555e7f00-b6e0-4aaa-ab47-a30478ef693b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154690 May 23 16:10:18.838284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1314/2520] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:19.673952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:19.674573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:19.681108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.681551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.682305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.682743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.699580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:19.700332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:19.707005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.707557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.708318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.708750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.863521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:19.872410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48727b8f-4e3e-41f0-8caa-5c7b3b966b41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.202255 May 23 16:10:19.873119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1207/2521] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:19.877710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:19.881069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:19.884943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.885414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.886143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.886723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.887911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:19.896687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:19.898211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:19.899407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fb9edb30-4a4f-4268-805b-8d01b8374d8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.204158 May 23 16:10:19.899969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1315/2522] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:19.904323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:19.904941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:19.910640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.921758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.921758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:19.921758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:19.952334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:19.960306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9fae4181-e82e-4239-8dc0-6d4b8a872097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086215 May 23 16:10:19.960903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1208/2523] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:20.047727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:20.056343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5581453b-1488-4181-8001-227ea2006ef8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155431 May 23 16:10:20.056919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1316/2524] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:20.889780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:20.890346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:20.897222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:20.897848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:20.898906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:20.899419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:20.919150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:20.919928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:20.926617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:20.927073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:20.927856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:20.928291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:21.113807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.117505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.123260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a47cab73-d8c6-4f13-97bd-41f285b728ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.237451 May 23 16:10:21.123963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1209/2525] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:20 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:21.126033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.127579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.128565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:21.128755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b27144ab-9bec-486b-81ff-efcafebfa273 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214891 May 23 16:10:21.129297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:21.133254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1317/2526] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:21.134344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:21.134969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:21.135773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:21.136299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:21.137144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:21.137651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:21.141064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:21.141758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:21.142265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:21.142527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:21.143038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:21.224196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.232771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a6687d5-6e50-412c-b1bf-09adc2d6780d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107748 May 23 16:10:21.239356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1210/2527] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:21.293347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:21.301312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e3379957-75db-413b-bc10-b6fdc2ddd5e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.170504 May 23 16:10:21.301886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1318/2528] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:22.140580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:22.141228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:22.147132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:22.147771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:22.148968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.149458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.150268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.150700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.155006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.155513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.156275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.156711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.347434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.350539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.356164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95b280b9-2e45-42cf-8e01-a253f30e783f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219479 May 23 16:10:22.356815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1211/2529] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:22.359651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.361145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:22.361262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.361748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:22.362315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae9d469e-9666-4f6d-8646-11979925e19b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219064 May 23 16:10:22.367571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1319/2530] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:22.367571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:22.367821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.368123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.368240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:22.368870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.369305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.373903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.374358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.375240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:22.375678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:22.455512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.464672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-00a66dca-76a7-4da8-ac1b-4707f77e725d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106910 May 23 16:10:22.465321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1212/2531] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:22.524021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:22.530402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6e4824c-9bb4-4b59-bbeb-2e253107b8c3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.166231 May 23 16:10:22.530962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1320/2532] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:23.374264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:23.374878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:23.381634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.382147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.382904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:23.383365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.383365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.383521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:23.389984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.390430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.391216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.392023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.577324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.581315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.585360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52018acb-2c90-4964-8b50-ceb4fd14c5de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214900 May 23 16:10:23.586799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1213/2533] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:23.590403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.591217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:23.591831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:23.591965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.593087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-54396398-1bdb-4749-83f5-59bccbaf07f0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214012 May 23 16:10:23.593809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1321/2534] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:23.598010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.598494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.599289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.599725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.599949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:23.600575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:23.607844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.608298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.609063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:23.609499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:23.669715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.677927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e2444362-22c8-40f0-a6a7-fc944e024e5e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090262 May 23 16:10:23.678525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1214/2535] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:23.756200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:23.764104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a86833a0-16b2-4212-bc4e-c4c0a4e78b59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.167787 May 23 16:10:23.764671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1322/2536] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:24.602935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:24.603582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:24.610177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.615247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.615247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.615247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.615582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:24.615582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:24.619696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.620147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.620909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.621339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.806591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.812702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.821256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.822836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.824033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fcc120fd-b048-4738-8d53-ec86952a825c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.215969 May 23 16:10:24.824232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6dddc36e-7070-4c0a-bd59-a41f4c377065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.225103 May 23 16:10:24.824714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1323/2537] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:24.833678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1215/2538] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:24.833839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:24.833929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:24.834010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:24.834107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:24.835990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.836215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.836467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.836669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.837227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.837437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:24.837662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.837915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:24.905417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.913428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a73e244-6c81-4138-a786-d36d50670e28 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087765 May 23 16:10:24.913996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1324/2539] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:24.987037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:24.995788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1bace561-56bb-4185-9cff-0d709318ea89 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169619 May 23 16:10:24.996358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1216/2540] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:24 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:25.841897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:25.842457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:25.842843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:25.843026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:25.849127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:25.849317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:25.849574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:25.849700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:25.850340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:25.850452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:25.850777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:25.850880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:26.043474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.046533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.051535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73a89db1-79cb-4166-af15-6d75c1904d4f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214050 May 23 16:10:26.052256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1325/2541] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:26.055001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.056573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.056768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:26.057493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:26.057793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1585139e-3900-4d2c-83e1-7f5816ceb60d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219707 May 23 16:10:26.062148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1217/2542] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:26.063459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:26.063917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:26.065071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:26.065190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:26.065521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:26.065718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:26.074633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:26.075087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:26.075909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:26.076348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:26.148477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.157011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b07fb985-fc92-4835-b856-e84cabc8b9b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.103771 May 23 16:10:26.157670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1326/2543] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:26.231888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:26.240118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a3b33f9-8e91-4980-8365-b415af948a75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.180319 May 23 16:10:26.240767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1218/2544] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:26 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:27.068605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:27.069223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:27.076015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.076466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.077253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.077364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:27.077760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.077944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:27.084356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.084811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.085569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.086008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.269009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.277618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47e1a250-af5b-446a-94ff-c19d2ecad3b8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212832 May 23 16:10:27.278196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1327/2545] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:27 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:27.280280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.282618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:27.283258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:27.289013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.289468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.289653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.290238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.290678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.291997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.293305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7f3db0bb-0999-4a2a-9bd0-874aa0c01ee4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219741 May 23 16:10:27.293956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1219/2546] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:27.298428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:27.299136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:27.304960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.305413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.306177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:27.306644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:27.354013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.362170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fccab7cb-b033-431c-a251-8c2171185dc0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082983 May 23 16:10:27.362749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1328/2547] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:27.444797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:27.452908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ee5a587-529a-48d3-885f-5b676baf595c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157842 May 23 16:10:27.453467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1220/2548] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:28.294287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:28.294915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:28.297275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=98 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:28.302072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.307062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.307062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.307062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.312527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:28.313292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:28.319964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.320415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.321175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.321609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.489915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.498823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9577b582-471c-40df-bdac-40ca67fd6268 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208412 May 23 16:10:28.499421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1329/2549] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:28 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:28.503805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:28.504408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:28.510564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.511017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.511884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.512315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.514162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.523770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.525360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.526530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abee480c-e8a4-4434-860c-54a14823cd63 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217889 May 23 16:10:28.527084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1221/2550] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:28.531416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:28.532026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:28.537810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.538260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.539017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:28.539481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:28.576769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.580171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:28.585067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56bce0af-1ba9-4b33-8bfa-7b9517691a61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084683 May 23 16:10:28.585667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1330/2551] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:28 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:28.672168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:28.680098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a2b5f3c-f2e6-4567-8d15-02368a7aa457 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152062 May 23 16:10:28.680654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1222/2552] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:29.108752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-366e6bce-daac-46b7-aa8a-316af68606a0 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] Calling method '.version_select at 0x7f47f9166f70>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:29.218437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-366e6bce-daac-46b7-aa8a-316af68606a0 tempest-TestVolumeBootPattern-1681705579 tempest-TestVolumeBootPattern-1681705579-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1169274407" status: 202 len: 0 microversion: 2.1 time: 0.113759 May 23 16:10:29.218958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1331/2553] 10.222.0.32 () {58 vars in 1262 bytes} [Mon May 23 16:10:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1169274407 => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:10:29.517132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:29.517775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:29.529663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:29.529663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.529663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.529663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.530040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.545996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:29.546615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:29.553197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.559063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.559063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.559063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.726609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.730627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.734562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e18fc15e-5c5e-4cfe-931b-ac00bcf3eb78 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.221565 May 23 16:10:29.735378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1223/2554] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:29.739470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.740995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.742177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5380ca1e-4f89-4e28-8dbb-bddab2a686c8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.200783 May 23 16:10:29.742881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1332/2555] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:29.747729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:29.748342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:29.749403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:29.750014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:29.755954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.756407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.756602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.756864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.757631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.758071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.761885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:29.765499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:29.856706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.865252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fc899318-7e44-4061-8984-b0f3e034e709 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.119433 May 23 16:10:29.865831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1224/2556] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:29.927209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:29.936000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3bb5054b-2e09-4706-b8fc-fa16acbb5d80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.192268 May 23 16:10:29.936588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1333/2557] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:30.753847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:30.754523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:30.761085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:30.770429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:30.770583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:30.770583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:30.770583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:30.770583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:30.770907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:30.770907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:30.770907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:30.770907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:30.983337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:30.986984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:30.992067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-218e9816-9795-4d59-a4eb-a2f44d1de345 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.243194 May 23 16:10:31.003497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1225/2558] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:31.015514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:31.015514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:31.015514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a2a370f-bdd1-4840-9de6-2c859da98e72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.252398 May 23 16:10:31.016362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:31.016466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1334/2559] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:31.016819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:31.016907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:31.017621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:31.020195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:31.021012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:31.022205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:31.022862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:31.024288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:31.024990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:31.025772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:31.030327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:31.157470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:31.161646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:31.168281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-762ba6f6-3de0-414c-85b5-97942dcb3f7b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.162836 May 23 16:10:31.173517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1226/2560] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:31.303754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:31.312168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-05beaaa0-d4af-44cc-bd86-54168303cc60 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.301538 May 23 16:10:31.315068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1335/2561] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:32.017970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:32.018711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:32.025299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.025869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.026773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.027240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.032868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:32.033630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:32.040224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.046034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.046034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.046034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.314732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.325319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-96c69bd3-1300-4978-9411-447f3ccfa380 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.311758 May 23 16:10:32.326398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1227/2562] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:32.327361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.331940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:32.332754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:32.337249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.339056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.339225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.339680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.340935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.341129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.342040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f599d9e3-8a3d-493d-9035-9ff9c033a589 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.313121 May 23 16:10:32.355451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1336/2563] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:32.360205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:32.360826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:32.366778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.367258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.368023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:32.368751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:32.451247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.461886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-767d38e8-9fcb-4d7c-9de8-1ddc18b40846 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.133911 May 23 16:10:32.462740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1228/2564] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:32.572832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:32.580766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-878a5ee7-065e-43a8-a67e-0eefd4bbf34e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.224429 May 23 16:10:32.581333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1337/2565] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:33.342115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:33.342725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:33.349181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.349628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.350374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.350804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.360568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:33.361213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:33.367999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.368451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.369222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.370197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.642920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.646238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.653232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0fd22e8-c616-47f9-9600-5c6af68d1d8a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.314968 May 23 16:10:33.653919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1229/2566] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:33.655556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.657238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.665579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-969da78d-53e9-4b2f-954e-cb2725191c0c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.302026 May 23 16:10:33.665579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1338/2567] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:33.665579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:33.665579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:33.669564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:33.670314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:33.671537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.672018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.673111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.673622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.676983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.677528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.678434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:33.678957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:33.797688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.806212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cedd7b07-d080-448a-a131-4c67f04f5983 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.151108 May 23 16:10:33.806905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1230/2568] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:33.858432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:33.866574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-38c4dd1d-ce3e-4b55-8f5e-3c718eab3f97 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.206232 May 23 16:10:33.867178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1339/2569] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:33 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:34.670660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:34.677494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:34.680868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:34.684774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:34.691747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.692245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.692902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.693340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.693874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.694362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.695199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.695646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.882550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:34.891325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02cf6b86-9ba7-4fe0-8d5d-e8783541d8ba tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.225172 May 23 16:10:34.901763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1231/2570] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:34 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:34.906792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:34.907643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:34.910661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:34.914044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.914494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.915486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.915922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.919896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:34.922105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:34.923510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d47517b5-2034-46b0-8dfe-d7066c2fe189 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.250180 May 23 16:10:34.924166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1340/2571] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:34.929162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:34.930021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:34.940025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.941000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:34.942225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:34.942906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:35.240690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:35.251990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4a264bc-5a0c-4742-bcb7-824ce6b04aae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.348843 May 23 16:10:35.263484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1232/2572] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:35.345937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:35.353729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff502f27-f037-4a75-b18f-80495299a554 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.428237 May 23 16:10:35.354328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1341/2573] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:35.908241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:35.908769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:35.915405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:35.915860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:35.917142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:35.917578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:35.942378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:35.943002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:35.950165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:35.950840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:35.951809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:35.952437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:36.112430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.121080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-abd1533a-689f-4f4b-bee4-5fe0c24db714 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216749 May 23 16:10:36.121757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1233/2574] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:36.138394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.147117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.148796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.149964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4d69a004-b6c3-408b-808d-2ed46e548aec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.211754 May 23 16:10:36.150519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1342/2575] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:36.269716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:36.270348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:36.276927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:36.277381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:36.278147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:36.278630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:36.339857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.348091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a63abb93-8a57-473a-8116-382eabeedebd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082541 May 23 16:10:36.348668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1234/2576] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:36.370255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:36.370891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:36.377160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:36.377616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:36.378390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:36.378833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:36.539911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:36.549335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b9e0fe3-ecf3-44e8-a433-47f02ab1c7cc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.183499 May 23 16:10:36.550462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1343/2577] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:37.138123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:37.138702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:37.145752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.146191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.147124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.147699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.168765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:37.169477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:37.176031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.176561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.177393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.177920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.358622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.362581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.366681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-92f3a1ae-b641-4685-8004-700ea3b8f173 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.232775 May 23 16:10:37.367287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1235/2578] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:37.371296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.372821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.373458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:37.374019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-493c704f-6eb9-468b-8f7b-183f05cad0bf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.209810 May 23 16:10:37.374271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:37.374578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1344/2579] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:37.380468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.380916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.381681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.382115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.442942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.451601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-05ba2e57-4dcc-431e-a4fe-8752e4e58375 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083372 May 23 16:10:37.452166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1236/2580] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:37.567371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:37.567970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:37.574408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.574969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.575693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:37.576131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:37.706677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:37.715665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-90449670-6a1d-4e92-a52f-e06dcf8c30af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152170 May 23 16:10:37.716238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1345/2581] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:38.384164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:38.384781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:38.392573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.394543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:38.395025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.395184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:38.395285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.395285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.399696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.400144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.400898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.401356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.590545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.594083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.599651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b8b1b45b-02a2-4132-a5bf-41d0b737bafd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219257 May 23 16:10:38.600284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1237/2582] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:38.605571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:38.606422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.606587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:38.609950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.609950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dd8a19bc-bb6a-4eb8-959a-5393b8309065 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220743 May 23 16:10:38.609950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1346/2583] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:38.616766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.617454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.618229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.618736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.687935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.695863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e67a2ccc-2ca6-4f7a-8fc8-95643a4c98fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.094469 May 23 16:10:38.696644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1238/2584] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:38.733373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:38.733996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:38.740000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.740453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.741217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:38.741659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:38.870514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:38.879564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e024718c-2c3e-408c-b704-03707f2e383b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150619 May 23 16:10:38.880378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1347/2585] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:39.617481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:39.618097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:39.625360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.625806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.626560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.626996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.627632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:39.628252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:39.634673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.635148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.635911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.636350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.654666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:39.820628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:39.830130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:39.836179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-74c7be71-ffad-4d32-bb30-a257c2012c31 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.222481 May 23 16:10:39.836767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1239/2586] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:39.839047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:39.840661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:39.841924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31ea2ad0-3fc0-4c9e-972b-11c8b5895b59 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.218207 May 23 16:10:39.842724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1348/2587] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:39.844242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:39.844860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:39.850862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.851354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.852159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.852661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.897552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:39.898076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:39.904216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.909515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.909515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:39.909515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:39.914579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:39.923309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-480bff09-a6f0-4452-92d0-3a15049546b3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082598 May 23 16:10:39.924091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1240/2588] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:40.036675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:40.045198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c47aa412-af67-4b31-9d79-7fae9ed3291f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151599 May 23 16:10:40.045765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1349/2589] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:40.854185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:40.854844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:40.860081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:40.860693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:40.861564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:40.862018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:40.862787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:40.863253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:40.867117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:40.867587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:40.868458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:40.870615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:41.060511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.064343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.068563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ab0d46cd-fc81-4d58-b0a7-5e209927c240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219136 May 23 16:10:41.069126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1241/2590] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:41.072779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.073538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:41.074146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:41.074277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.075442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73ede58d-7933-432c-919d-2e3b1247a22d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219374 May 23 16:10:41.076159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1350/2591] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:41.080622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:41.081225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:41.085757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:41.086398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:41.086976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:41.087356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:41.087468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:41.087796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:41.088703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:41.089126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:41.161993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.169843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9f552a4-1e98-40bf-92bb-c7b9002a7860 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.099785 May 23 16:10:41.170403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1242/2592] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:41.251134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:41.259123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e71c14ba-1ef6-417d-af94-ce97e1ee27ea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.181887 May 23 16:10:41.259707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1351/2593] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:42.085571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:42.086185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:42.093372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.093819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.094104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:42.094636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.094744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:42.095078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.101576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.102020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.102766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.103225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.112700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:42.284872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.291621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.292896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5f05adc3-6fc3-4d69-a4b1-22fc3599c05d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211148 May 23 16:10:42.293476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1243/2594] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:42.297958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:42.298542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:42.301450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.303248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.304305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.304511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-270f536b-7b28-4fb0-88d8-2ad1ddbbfcdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214858 May 23 16:10:42.304758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.305072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1352/2595] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:42.305516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.305954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.310148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:42.311011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:42.316949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.317397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.318144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:42.318574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:42.368917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.377335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-97023d51-8c2f-4afc-bbb8-4a14c0a7c201 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082895 May 23 16:10:42.377905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1244/2596] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:42.465316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:42.473227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-efbad853-7eac-43ca-8d8a-f20f92920ece tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.167224 May 23 16:10:42.473793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1353/2597] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:43.310012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:43.310634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:43.317352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.317784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.318553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.318993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.322460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:43.323072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:43.329969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.330409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.331209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.331641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.514381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.518180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.522735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2ea4cf3e-7c4e-411b-9082-1df8a123299f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216537 May 23 16:10:43.523298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1245/2598] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:43.526704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.528428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:43.528898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.529067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:43.530084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de758246-f5da-448d-877f-1bed90782499 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.211428 May 23 16:10:43.530767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1354/2599] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:43.535141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.535291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:43.535601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.535836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:43.536377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.536815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.541861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.542311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.543078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:43.543536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:43.608063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.616021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07f3c65e-1c56-4cc1-9c05-1f0d6247176e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091774 May 23 16:10:43.616872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1246/2600] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:43.692731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:43.701504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5138e11-eff6-49ec-8be2-a03245e05fce tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169909 May 23 16:10:43.702074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1355/2601] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:44.553084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:44.553967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:44.554430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:44.554963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:44.564608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.565137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.565327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.565890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.566579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.567161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.567333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.567878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.776666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.781488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.785674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.787205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.788375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0bacc40b-7e26-4e84-b1d0-4a5a1aa1801e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.241143 May 23 16:10:44.788934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1356/2602] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:44.789813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c66821b2-e547-4322-a74f-eec50dfc2583 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.243242 May 23 16:10:44.790601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1247/2603] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:44.794023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:44.794633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:44.795578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:44.796298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:44.800688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.801137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.801893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.802327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.802460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.803174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.803950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:44.804472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:44.892873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.901153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08ff6b71-1924-449d-b5de-61425ae7cb8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.111264 May 23 16:10:44.901961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1357/2604] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:44.959648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:44.968595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe26741f-bb28-4fa3-b40d-c9b21c055779 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.177168 May 23 16:10:44.969154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1248/2605] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:45.815643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:45.816283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:45.816283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:45.816767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:45.822929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:45.823220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:45.823413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:45.823669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:45.824239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:45.824495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:45.824600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:45.824925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:46.013649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.020569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.021903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4a01379-f739-4cdd-bcbd-ff11d50711ad tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210133 May 23 16:10:46.022668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1358/2606] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:46.026866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:46.028027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:46.029825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.031366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.032567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1134dc57-f5ce-4251-bf04-d0f109143caf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220838 May 23 16:10:46.033120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1249/2607] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:46.034571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:46.035077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:46.035884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:46.036475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:46.037448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:46.038291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:46.044828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:46.045274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:46.046028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:46.046457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:46.122436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.130301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d504216d-2c23-460c-befe-d70023121685 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106870 May 23 16:10:46.130877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1359/2608] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:46.189522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:46.197359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9626e2aa-2b7e-465b-909c-f7f613ad44fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163317 May 23 16:10:46.197920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1250/2609] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:47.039535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:47.040150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:47.046632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.047084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.047882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.048380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.053711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:47.054447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:47.062092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.067766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.067766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.067766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.250431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.252960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.258393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7038bb18-5ae7-4a19-8aff-154c77a4bd98 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.222661 May 23 16:10:47.258936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1360/2610] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:47.262356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.263475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:47.264127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:47.264765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.266042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2cd05509-ecb7-4699-abf5-6916ebd62084 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.216277 May 23 16:10:47.267532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1251/2611] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:47.270720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.271237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.271694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:47.272312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:47.278380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.279017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.279177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.279519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.280284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:47.280722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:47.366541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.375183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-927bc475-55ac-4ea4-875b-915515c0d593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.115244 May 23 16:10:47.375770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1361/2612] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:47.434113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:47.442143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-310ac283-60f6-46ad-8b70-9ab69c5ab408 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.174336 May 23 16:10:47.442730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1252/2613] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:48.279849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:48.280463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:48.287184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.287632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.288381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.288820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.289672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:48.290292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:48.296721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.297174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.297950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.298389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.475506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.483446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-61e98ee5-6e28-44db-82ad-a4e1daeb7f02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.207454 May 23 16:10:48.484340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1362/2614] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:48 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:48.486990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.488609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:48.489215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:48.495256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.495708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.496384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.496990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.497422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.497811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.498969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b7e87d48-e9b8-4c9a-8ff0-bc1a8132719f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.213286 May 23 16:10:48.499551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1253/2615] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:48.503829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:48.504430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:48.510054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.510493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.511273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:48.512532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:48.562440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.570754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b47f754-4c1d-4c77-968f-c9005812874c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085573 May 23 16:10:48.571364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1363/2616] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:48.654584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:48.662584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-79b81a21-fa83-40bb-a20f-dfd93988be69 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162078 May 23 16:10:48.663174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1254/2617] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:49.500024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:49.500674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:49.507073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.507554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.508297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.508727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.517308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:49.525003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:49.525003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.525003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.525509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.525961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.709685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.712710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.718252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-68b97626-bf82-43a7-9014-10fcfffba813 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.222118 May 23 16:10:49.719201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1364/2618] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:49.721976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.723624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.723913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:49.724528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:49.725184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-358350b7-f2ca-40ae-a429-cfd68d753ed4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.211779 May 23 16:10:49.726052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1255/2619] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:49.731529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.732194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.733174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.733885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.734027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:49.734594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:49.738360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:49.740462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.740912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.741673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:49.742111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:49.822920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.840654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dcea45a1-4a13-46d8-84a1-78dd78c737dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.120320 May 23 16:10:49.841414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1365/2620] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:49.889729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:49.898262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9ebc7b7-3af3-44d4-a508-875780159957 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.168418 May 23 16:10:49.898832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1256/2621] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:50.735274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:50.735892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:50.742384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.742833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.743625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.743851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:50.744260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.744506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:50.751525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.752026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.752791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.753233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.934480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:50.938345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:50.942603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96413f1d-f72b-4f5a-bf4b-66066007fc97 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211173 May 23 16:10:50.943194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1366/2622] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:50.946959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:50.947634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:50.948244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:50.948510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:50.949751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ec73ac6c-5795-4607-a19c-5e5e1edb435d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.209710 May 23 16:10:50.952225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1257/2623] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:50.954068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.954519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.954758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:50.955303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.955415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:50.955742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.961231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.961686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:50.962449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:50.962889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:51.038272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:51.046673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-48532df4-0300-4463-b12a-54c4601106c2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.102523 May 23 16:10:51.047263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1367/2624] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:51.103741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:51.112130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-324f6873-a815-4c9a-a82b-dddd67052115 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160877 May 23 16:10:51.112695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1258/2625] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:51.959715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:51.960335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:51.967339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:51.967792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:51.968551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:51.968819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:51.969272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:51.969448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:51.975939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:51.976398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:51.977158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:51.977595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:52.159794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.163057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.168415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-256d8655-d577-4e6d-9dad-b2f85b0231db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212019 May 23 16:10:52.169315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1368/2626] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:52.174068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:52.174282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.174679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:52.176187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.177513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f66fa74f-995f-48e5-bf60-cd358b27cfec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212990 May 23 16:10:52.177854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1259/2627] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:52.180579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:52.181026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:52.181786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:52.182221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:52.182429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:52.183053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:52.190287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:52.190736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:52.191536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:52.191975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:52.195971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:52.254487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.262477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77e4f08b-5a04-4aff-a177-a071d9136646 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092108 May 23 16:10:52.263058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1369/2628] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:52.337461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:52.346073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cfeddd7e-153a-4a4c-9eaa-d18a7e9f61f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.167256 May 23 16:10:52.346642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1260/2629] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:53.186228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:53.186856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:53.193541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.193994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.194762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.195232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.195440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:53.196063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:53.202426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.202880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.203677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.204116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.396910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.399715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.405061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4d08150-9593-4088-913e-588b07b17bc4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.223437 May 23 16:10:53.405640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1370/2630] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:53.408142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.409824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.410097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:53.410712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:53.410998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f5c798e-669a-4ba9-9eb0-81d9cd0a4002 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219558 May 23 16:10:53.415181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1261/2631] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:53.416179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:53.416706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.416824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:53.417161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.418438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.418993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.423190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.423710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.424548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:53.425180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:53.500937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.508855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-59886077-81b5-4b4e-a163-64ce82663495 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.102257 May 23 16:10:53.509639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1371/2632] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:53.566155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:53.574030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb75bb02-d3bf-4e6f-a8d1-2b717942b352 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161239 May 23 16:10:53.574570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1262/2633] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:54.422740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:54.423315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:54.429954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:54.430395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.430395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.430608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:54.431186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.431651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.436806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.437263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.438028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.438468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.623033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.629742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.631543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.633098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.634274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d5a7c1e1-9b3f-4e31-80c8-c7c9da4dee70 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.208312 May 23 16:10:54.634836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1263/2634] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:54.638649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2139552-d022-46b6-97df-c53a29d06ad8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220358 May 23 16:10:54.639522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1372/2635] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:54 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:54.644372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:54.644747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:54.645157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:54.645906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:54.651378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.651830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.651967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.652934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.653231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.653664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.653784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:54.654228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:54.724992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.732911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd6df067-400f-46f5-97bd-30b59d1600ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.097096 May 23 16:10:54.733497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1264/2636] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:54.805772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:54.813699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f35bc730-bfa6-4bfb-96f9-79b0149c1da1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173159 May 23 16:10:54.814271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1373/2637] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:54 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:55.654133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:55.655049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:55.655871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:55.656487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:55.661199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.661713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.662480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.662923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.663212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.663662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.664443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.664873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.855170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:55.860028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:55.864169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:55.865664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:55.866828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f4d1481c-a5ae-42fa-9394-a04120f1d96a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217980 May 23 16:10:55.867415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1265/2638] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:55.867895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0324a13-66f2-4eb9-87b4-cb19a195ddef tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216489 May 23 16:10:55.877447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1374/2639] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:55.877618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:55.877618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:55.877816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:55.877816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:55.879354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.879804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.880308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.880564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.880757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.881000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.881508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:55.881938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:55.965326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:55.973436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99400383-f43a-4996-bd48-749bb81187b6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.105062 May 23 16:10:55.974008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1266/2640] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:56.031454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:56.039386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0307002c-fe1d-4dd8-8e76-73e4b4edb6cd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169860 May 23 16:10:56.039954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1375/2641] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:56.885870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:56.887051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:56.888608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:56.889269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:56.893972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:56.894447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:56.895251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:56.895684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:56.898614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:56.899069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:56.899878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:56.900426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:57.086857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.091479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.096018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.097502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.098665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de6e4ae3-0751-497d-9d6c-cb6a9a02e11f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.216750 May 23 16:10:57.099234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1376/2642] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:57.099369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83017033-d0b6-4aab-8611-cd7fd14c8c26 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217862 May 23 16:10:57.099992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1267/2643] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:57.104191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:57.104841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:57.110394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:57.110843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:57.111635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:57.112071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:57.112406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:57.113000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:57.118750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:57.119229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:57.119972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:57.120402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:57.208414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.216895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9e25cd6d-fdb6-426e-9de7-221b419b35a1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.116704 May 23 16:10:57.217465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1377/2644] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:57.274437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:57.283367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6441c6ab-5346-42a4-8086-230a9b0bb222 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.182347 May 23 16:10:57.283927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1268/2645] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:58.116533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:58.117382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:58.118025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:58.118150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:58.119523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:58.123750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.124136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.124305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.124583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.125209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.125346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.125764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.125868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.262086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.270894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.272473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.273895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7804d052-5193-44a5-94c2-35f804bb271d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.160997 May 23 16:10:58.274490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1269/2646] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:10:58.279671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:58.280283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:58.288367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.289180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.290182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.290713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.471320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.479150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f3892c78-90b7-4254-805e-a0dd1aba286c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.203700 May 23 16:10:58.479719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1270/2647] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:58.484279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:58.484914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:58.490834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.491338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.492094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:58.492524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:58.528308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.536521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fa307ed3-a64b-4246-8c47-533b1b79ddd2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.424009 May 23 16:10:58.537198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1378/2648] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:58.629027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:58.637506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-38831f09-6cf5-4a95-80a3-cf510ccd7641 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.156610 May 23 16:10:58.638071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1271/2649] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:59.293090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:59.293690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:59.300143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.300594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.301338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.301767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.432197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.441047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.442548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.443758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dc64b490-bb66-4a68-a42c-100efe970f05 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.154629 May 23 16:10:59.444317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1379/2650] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:10:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:10:59.495695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:59.496728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:59.503069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.503542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.504291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.504721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.554135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:59.554762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:59.561322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.561780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.562544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.562984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.566014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.577934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5300bf14-a4ba-49d5-881f-a1d748ceaee5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086091 May 23 16:10:59.578515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1272/2651] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:10:59.653708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:10:59.654456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:10:59.656908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:10:59.661201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.661802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.664749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:10:59.664749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:10:59.695117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.703910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2741814d-5ea2-4e23-b8ca-ade110e4dbc9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.153773 May 23 16:10:59.704575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1380/2652] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:10:59.789383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:10:59.798081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-747cea97-8dfd-4af2-b1af-68aa7ed825e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148228 May 23 16:10:59.798640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1273/2653] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:10:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:00.462555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:00.463087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:00.470427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.470872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.471657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.472091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.597401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:00.598030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:00.604911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.605411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.606176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.606614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.608967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.612427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:00.617799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.619319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.620478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-91ef324d-9036-4ca7-8a23-afa5d588095b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.161967 May 23 16:11:00.621027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1381/2654] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:00.667918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.675694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b2d2ad8-af6b-4eed-a42b-f0108cea7d58 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083722 May 23 16:11:00.676230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1274/2655] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:00.721523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:00.722044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:00.728341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.728782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.729528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.729958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.815066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:00.815794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:00.821905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.822365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.823144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:00.823583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:00.857603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.865580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fdb855bf-30b7-416a-af90-929aae799802 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147902 May 23 16:11:00.866137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1382/2656] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:00 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:00.956619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:00.964492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fcf7f775-b3a3-4a93-9413-02116918f8b0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.153369 May 23 16:11:00.965055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1275/2657] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:00 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:01.638641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:01.639356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:01.646159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.646642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.647459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.647895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.692350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:01.692971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:01.699576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.705337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.705337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.705337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.771058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:01.774445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:01.780093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aeb9ab05-5cd2-4d7d-a14d-36f59f0e6476 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091691 May 23 16:11:01.780670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1276/2658] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:01.803461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:01.809227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:01.810768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:01.811967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8e7a956-62b0-433a-902e-ebf4589bc054 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.177447 May 23 16:11:01.812629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1383/2659] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:01.881789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:01.882406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:01.888826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.889221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.889974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.890412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.982288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:01.983033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:01.989080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.989636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:01.990461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:01.990964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:02.024602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.033028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-427c82bd-24e4-4843-9b31-25d923e2dee9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.155067 May 23 16:11:02.033605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1277/2660] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:02.116612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.124514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-34e87540-64a5-45e7-b34e-72be39a2d225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146707 May 23 16:11:02.125089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1384/2661] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:02.797330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:02.797973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:02.804686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:02.809316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:02.809316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:02.809316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:02.835290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:02.835290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:02.838880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:02.839564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:02.840432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:02.840976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:02.874072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.883237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b2c00857-a67c-4bd0-b25b-4d2939aaa09a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090083 May 23 16:11:02.884518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1278/2662] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:02.974930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.989703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.989703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:02.989703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0590d959-6a13-404a-8cd5-96d460dd1154 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.160967 May 23 16:11:02.990099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1385/2663] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:11:03.050195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:03.050816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:03.056741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.057193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.057952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.058390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.144913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:03.144913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:03.148980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.149433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.150194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.150623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.186082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:03.194282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e58d5e24-e8b7-4937-bd67-3601ed405e44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147909 May 23 16:11:03.194879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1279/2664] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:03.286043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:03.294749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-075f7120-1e50-4e9a-abb0-0e914f016e8a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.156590 May 23 16:11:03.295348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1386/2665] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:03.900791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:03.901345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:03.908586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.909036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.909806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:03.910245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:03.970902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:03.979260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-041f1829-d179-44e1-b18b-cf2a8c4a8b12 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082307 May 23 16:11:03.979819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1280/2666] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:04.005846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:04.006487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:04.014203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.014649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.015440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.015872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.146023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:04.154593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:04.156146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:04.157306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74fdf663-2cca-47aa-ab5c-b7a77997a653 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.155433 May 23 16:11:04.157856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1387/2667] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:04.214779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:04.215433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:04.221539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.221991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.222762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.223236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.311782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:04.312561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:04.318439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.318892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.319692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:04.320133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:04.354239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:04.363120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7638e7f5-e406-439f-bf57-e630ad4e2e5b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152272 May 23 16:11:04.363724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1281/2668] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:04.451030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:04.458901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7340af02-6177-4845-80e6-01dbf305c184 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151623 May 23 16:11:04.459486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1388/2669] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:04.996299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:04.996936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:05.004195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.004647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.005415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.005905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.071124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.079168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a6a9e89-c904-45e6-9ee0-012a0fac94f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086737 May 23 16:11:05.079726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1282/2670] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:05.177354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:05.177974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:05.184507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.185567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.186331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.186772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.319576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.328433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.329950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.331141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20933926-93ad-4ba1-8efc-01673e12ebdf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.158173 May 23 16:11:05.331698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1389/2671] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:05.380940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:05.381705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:05.388559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.389010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.389779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.390215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.475796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:05.476455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:05.482886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.483379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.484151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:05.484591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:05.523695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.532030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72fc9ca7-4111-4ba8-bfcf-6579c20a42cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.154979 May 23 16:11:05.532602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1283/2672] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:05.618975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:05.627640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aefbfd40-a4c8-4910-a5f1-d6ed0c89b113 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155700 May 23 16:11:05.628355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1390/2673] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:06.098203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:06.098894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:06.105360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.105807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.107066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.107523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.351953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:06.352512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:06.359743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.360200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.360974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.361415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.438572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.449015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dba207a4-1468-4d72-a67a-4afb3286b7f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.355922 May 23 16:11:06.449698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1284/2674] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:06.531177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.537735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.539331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.540486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-438442b4-5ebe-4aa4-ba61-ff6245e21a66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.192710 May 23 16:11:06.541033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1391/2675] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:06 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:11:06.549043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:06.549653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:06.555882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.556330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.557333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.558060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.644923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:06.645543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:06.652173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.652657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.653431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:06.653870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:06.703610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.713474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2153f12c-91ad-4fc4-ab40-abd9db5953a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.168228 May 23 16:11:06.714031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1285/2676] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:06 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:06.785458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:06.793791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8413c398-8a7d-44d0-8d61-8a21e3dd2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152895 May 23 16:11:06.794877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1392/2677] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:07.469553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:07.470129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:07.476726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.477173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.477932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.478363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.544685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.555553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6659bc39-31cf-4eeb-a2c8-685d5a4120d4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092078 May 23 16:11:07.557333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1286/2678] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:07.559249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:07.559896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:07.566367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.566816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.567593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.568032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.697846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.706772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.708314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.709481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6c2de8b-a1d7-48ec-a400-93c511b7bbd6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.154445 May 23 16:11:07.710039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1393/2679] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:11:07.730181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:07.730709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:07.736755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.737202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.737951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.738381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.811728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:07.812487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:07.818620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.819161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.819935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:07.820369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:07.898662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.906692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d19835aa-f8f4-4c80-8310-be482089ce71 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.180400 May 23 16:11:07.907293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1287/2680] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:07.962706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:07.970708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ceeaf7dd-9444-4669-87c6-0df3db2996c4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163400 May 23 16:11:07.971303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1394/2681] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:08.572248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:08.572920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:08.579530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.579967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.580727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.581162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.641895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:08.650340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9cd2aaf3-2390-46bf-8c02-401adcc4316e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081987 May 23 16:11:08.650913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1288/2682] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:08.728019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:08.728711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:08.736001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.736450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.737222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.737657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.866391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:08.874770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:08.876302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:08.877502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2bbe399e-507e-4884-bc6f-78d993f3d1d8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.153623 May 23 16:11:08.878060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1395/2683] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:08.923946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:08.924577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:08.931272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.931727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.932505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.932945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.987594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:08.988326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:08.994488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.995006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:08.995860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:08.996363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:09.112815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:09.120902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-00b1f056-9391-4866-b059-460ccafee82e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.201076 May 23 16:11:09.121659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1289/2684] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:08 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:09.158639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:09.166858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aa208bad-17ee-4cf1-a1ab-9564bfb1ab62 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.183040 May 23 16:11:09.167464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1396/2685] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:09.667942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:09.668570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:09.675597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:09.676051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:09.676808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:09.677241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:09.739953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:09.747950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7e588b64-ab8d-4604-a843-4bd73a7e2e2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083911 May 23 16:11:09.749103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1290/2686] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:09.896198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:09.896841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:09.903663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:09.904110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:09.905408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:09.905872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.044946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.053345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.054940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.056215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-710b8577-dbb3-43ca-ab34-e276ee51aebd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.164245 May 23 16:11:10.056768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1397/2687] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:10.137949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:10.138574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:10.145568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.146016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.146782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.147253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.183801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:10.184426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:10.191035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.191522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.192292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.192734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.211262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:10.333021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.341465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-338e7d51-b36f-4163-8984-6b1b599f107d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.207373 May 23 16:11:10.342159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1291/2688] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:10.359651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.367696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b15f222b-9586-4c08-9b5f-b65425228183 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.187712 May 23 16:11:10.368260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1398/2689] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:10 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:10.764705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:10.765329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:10.772231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.772685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.773450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:10.773876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:10.836867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:10.845575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c7345d85-ee66-48b0-9a36-edaa059095f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084749 May 23 16:11:10.846148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1292/2690] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:11.075792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:11.076365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:11.083379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.083835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.084624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.085062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.213056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.221883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.223398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.224668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45a33ee8-809b-4d3f-a812-900d5ddb3225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.152801 May 23 16:11:11.225301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1399/2691] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:11.358794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:11.359362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:11.365967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.366421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.367215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.367655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.385351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:11.386104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:11.386859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:11.392570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.393020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.393789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.394278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.566461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.574420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6cce865e-0c5e-4187-9e66-b422ed665ee0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219570 May 23 16:11:11.575000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1293/2692] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:11.577585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.586638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8d691b8-0128-46fd-8775-5ff9ea650387 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.205224 May 23 16:11:11.587497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1400/2693] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:11.863561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:11.864162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:11.870729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.871244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.871960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:11.872394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:11.937051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:11.945357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-29ae787e-c80b-468d-8ad2-c3256eee4e4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085677 May 23 16:11:11.945932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1294/2694] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:12.243750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:12.244366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:12.251841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.252293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.253123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.253530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.387606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:12.396824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:12.398347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:12.399558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3921d25e-9ded-453b-b70c-05635ffb875d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.159657 May 23 16:11:12.400114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1401/2695] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:12.592118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:12.592738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:12.598749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.599236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.600029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.600463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.603155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:12.603787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:12.609712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.610670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.611468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.611936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.792394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:12.796475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:12.801231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e30b65a8-f385-41dc-9061-df6c25cb7197 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212931 May 23 16:11:12.801811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1295/2696] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:12.804953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-221dab9c-6504-4e1f-9348-47c420eb91aa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.205583 May 23 16:11:12.805514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1402/2697] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:12 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:12.962786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:12.963358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:12.970318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.970780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:12.971556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:12.971990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.033119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:13.040881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1b6f162b-3cbc-4e95-b5e3-91c5047269ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081978 May 23 16:11:13.041442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1296/2698] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:13.417840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:13.418454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:13.425055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.425503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.426254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.426689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.557675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:13.566734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:13.573198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:13.573198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bcd53f19-0f21-4152-b727-af14c9c35b8d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.155520 May 23 16:11:13.573198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1403/2699] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:13.818948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:13.819524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:13.821385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:13.822039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:13.825588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.826030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.826787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.827269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.827826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.828277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:13.829630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:13.830066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:14.018879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.022490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.026955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b2dcec8-801b-4c0a-bb1c-ec06a1b5bb00 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211935 May 23 16:11:14.031482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1297/2700] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:13 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:14.031608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8fbb395b-c788-4d93-b8fd-9a1907fd2e1c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212666 May 23 16:11:14.031608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1404/2701] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:14.057623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:14.058260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:14.064927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:14.065387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:14.066132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:14.066557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:14.130789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.138760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0bb2f180-325a-4ff7-a37d-5c0b8d4cff78 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085027 May 23 16:11:14.139403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1298/2702] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:14.588331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:14.588973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:14.595936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:14.596413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:14.597169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:14.597606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:14.730196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.738528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.740640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:14.741205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36603418-0018-498c-beb4-884491ea4057 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.156792 May 23 16:11:14.741759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1405/2703] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:15.044551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:15.045153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:15.046821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:15.047470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:15.052439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.052884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.053671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.054016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.054129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.054472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.055258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.055695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.239350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.258325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b01c88b-5dc6-4171-bfcb-dfe4dc7d101d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.206999 May 23 16:11:15.258325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1299/2704] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:15.258594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.258716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:15.258716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:15.262366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.262843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.263470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ef9b813d-5cf7-4cae-a0c3-8b054af237c2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.220233 May 23 16:11:15.263664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.264061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1406/2705] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:15.264172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.331473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.342304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4a309b8-e17e-40d4-983c-65895e3b828e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092961 May 23 16:11:15.342896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1300/2706] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:15.760041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:15.760677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:15.767473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.767939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.768686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:15.769116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:15.897270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.905851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.907881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:15.909039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-01f82025-4187-4560-b752-b99a915f77ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.152996 May 23 16:11:15.909593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1407/2707] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:16.265559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:16.266182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:16.273475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.273927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.274685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.275160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.281474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:16.282721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:16.288580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.289022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.289775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.290275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.468662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:16.476126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:16.477360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4b7354c2-0606-404e-bca4-c620e6bb449d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215592 May 23 16:11:16.478052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1301/2708] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:16.482404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:16.483013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:16.484208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bda05b8e-b411-44c5-ba16-1b09a5cbb958 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.206491 May 23 16:11:16.484904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1408/2709] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:16 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:16.489437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.489892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.490663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.491118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.551893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:16.559755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25e5b6fa-8981-45d6-8ae0-973835930ae6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080888 May 23 16:11:16.560321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1302/2710] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:16.928019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:16.928637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:16.935180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.935634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:16.936386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:16.936821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.062693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.071234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.072743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.073911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-87887806-bfeb-4531-96e6-658214c793a8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.149761 May 23 16:11:17.074467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1409/2711] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:17.494981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:17.495624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:17.501164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:17.501763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:17.502358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.502809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.503596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.504037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.507615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.508062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.508807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.509236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.693978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.697977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.702073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a74b85e-f62b-4661-9b46-f12601ffd27a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210980 May 23 16:11:17.702644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1303/2712] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:17 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:17.706487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e68780a-5900-4be7-ad18-a5bf3d60b77f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.209567 May 23 16:11:17.711142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1410/2713] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:17.711301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:17.711301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:17.714741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.715211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.715961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:17.716394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:17.776385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:17.784092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1f43f3f1-e913-42a3-a10c-0894d025faac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080492 May 23 16:11:17.784655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1304/2714] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:18.092336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:18.092876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:18.100010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.100460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.101211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.101648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.229486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:18.238052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:18.239637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:18.240810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ebd5bda-d83e-427f-b6d7-8bf714114094 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.152469 May 23 16:11:18.241366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1411/2715] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:18.719737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:18.720288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:18.723060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:18.723736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:18.726813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.727303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.728059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.728494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.729571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.730028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.730790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.731259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.920435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:18.924227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:18.929031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e517c41f-095b-4ffa-a602-e7d5a32718ff tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213220 May 23 16:11:18.929695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1305/2716] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:18 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:18.932336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe37ad58-0889-4b75-a378-2142fc3fca1f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213105 May 23 16:11:18.933045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1412/2717] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:18.934036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:18.934674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:18.942192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.942643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:18.943432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:18.943864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.009754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:19.018671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ce19c8d6-23fe-4c9d-a968-4c007e1705dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088012 May 23 16:11:19.019405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1306/2718] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:19.259620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:19.260250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:19.269263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.269743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.270966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.271589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.405657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:19.414465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:19.415967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:19.417156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8cd4cda0-c923-4014-aa22-0b5440db8577 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.161524 May 23 16:11:19.417708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1413/2719] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:19.946147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:19.946796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:19.949013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:19.949631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:19.953378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.954408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.955218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.955657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.956236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.956681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:19.957435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:19.957876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:20.145229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.148716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.153577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe30073a-943c-4f34-a40b-6f95e5304dca tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211255 May 23 16:11:20.154178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1307/2720] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:20.156806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-268c9a12-68be-45db-adff-b0138692a876 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.211527 May 23 16:11:20.157593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1414/2721] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:20.158854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:20.159502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:20.165784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:20.166233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:20.167021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:20.167484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:20.237759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.245825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6df00679-36a6-44e6-b305-9727faff0251 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090723 May 23 16:11:20.247722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1308/2722] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:20.435595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:20.436201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:20.442649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:20.443344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:20.444096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:20.444523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:20.448391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:20.570932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.579309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.580800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:20.581944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf84c72a-52cf-4476-a73d-cd51a4eeb24d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.150273 May 23 16:11:20.582493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1415/2723] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:21.170378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:21.171121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:21.178573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.179029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.179951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.180388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.230426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:21.231194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:21.237996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.238523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.239401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.239932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.385005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.394467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d63b97a5-0ea5-4839-90dd-5d47a5b172be tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.228377 May 23 16:11:21.404896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1309/2724] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:21 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:21.404896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:21.404896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:21.406682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.407689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.408550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.408984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.414191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:21.433939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.433939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae1a8d3d-9984-4a4c-8dbf-9ab568cb5813 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.204645 May 23 16:11:21.433939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1416/2725] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:21.471189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.479265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-769f3ec0-43c6-4392-98bd-8a028012b834 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082953 May 23 16:11:21.479885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1310/2726] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:21.599388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:21.599992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:21.606396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.606860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.607838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:21.608275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:21.739686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.747970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.749444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:21.751304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-383ced3c-fe54-4351-9edf-0bb50b45c4c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.155215 May 23 16:11:21.751858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1417/2727] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:22.412834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:22.413456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:22.419999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.420451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.421206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.421646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.449821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:22.450367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:22.457001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.457452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.458215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.458651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.604479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.612663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-433f3e3c-9ce9-44c7-b6a0-7734ba83be85 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.203699 May 23 16:11:22.613534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1311/2728] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:22.617782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:22.618421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:22.624301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.624748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.625503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.626340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.626824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.634309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-18f7116e-8d72-4412-84da-6450bde1dcf0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.188410 May 23 16:11:22.634872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1418/2729] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:22.686407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.694740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6db1d9b9-fa4b-43f0-a6ab-7c9d52f0c2fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080432 May 23 16:11:22.695332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1312/2730] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:22.769668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:22.770284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:22.776274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.776720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.777470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:22.777906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:22.910018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.918346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.919883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:22.921042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-686e625b-3237-4692-a8bb-471894d2580b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.155496 May 23 16:11:22.921599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1419/2731] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:23.630337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:23.630989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:23.637482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.637938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.638693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.639154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.651076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:23.651707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:23.658446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.663805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.663805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.663805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.837976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:23.841994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:23.846037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-71a42b7e-786a-4643-ac47-01b2ed02fd34 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220124 May 23 16:11:23.846604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1313/2732] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:23.849955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b203438a-ef51-4e30-a348-f7241249b354 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.202590 May 23 16:11:23.850661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1420/2733] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:23.853472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:23.854173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:23.860580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.861036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.861805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.862243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.924448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:23.932676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bbc87e2b-1215-405b-a4e6-e6851be71746 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085140 May 23 16:11:23.933246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1314/2734] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:23.940913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:23.941520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:23.947432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.947876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:23.948624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:23.949052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:24.077476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:24.085748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:24.087270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:24.088423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-276da08b-6078-41dd-a71a-1dcf3f2d8df7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.151978 May 23 16:11:24.088974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1421/2735] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:11:24.863089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:24.863793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:24.866679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:24.867343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:24.870639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:24.871085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:24.871880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:24.872313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:24.873698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:24.874169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:24.874917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:24.875383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:25.067700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.070738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.075728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77176f63-b269-494f-b1b1-73377b7c614d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216496 May 23 16:11:25.076357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1315/2736] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:25.078637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c942ee10-dddb-44fd-ac00-c128f41bba72 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.215848 May 23 16:11:25.079968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1422/2737] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:24 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:25.081396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:25.082012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:25.087814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:25.088268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:25.089031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:25.089470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:25.106577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:25.107223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:25.113396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:25.113854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:25.114628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:25.115204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:25.156558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.166740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0c89bf4d-b06c-4dc7-9ef8-314bdf69d194 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089412 May 23 16:11:25.168209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1316/2738] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:25.251377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.259904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.261413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:25.262583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a4201484-cde6-43c5-ad51-e745a8f4f3b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.159841 May 23 16:11:25.263214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1423/2739] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:11:26.100706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:26.101359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:26.101359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:26.101859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:26.107782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.107935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.108231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.108354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.108982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.109098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.109411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.109514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.302070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.305304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.310078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32a8a358-fd77-4327-9814-21174c3c3b04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213201 May 23 16:11:26.310916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1424/2740] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:26.315814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:26.316418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:26.317516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb55a2ef-ab96-4de7-b429-0ea4cc1fcddc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.220586 May 23 16:11:26.320516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1317/2741] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:26 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:26.322355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.322743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:26.322895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.323493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:26.323626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.323978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.329338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.329783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.330532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:26.330998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:26.391183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.399477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14012b6f-87f3-4bb7-a8a1-fc22be1908a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087692 May 23 16:11:26.400047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1425/2742] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:26.475384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.483945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.486215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:26.487452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf408f98-b985-4eb7-bf16-c6cb93e1f56b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.168334 May 23 16:11:26.488097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1318/2743] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:27.328193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:27.329576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:27.334579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:27.335229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:27.336105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.336548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.337335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.337766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.341368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.341815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.342572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.343010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.534661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.538867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.542779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73e14ed5-4fdd-49cc-90ab-ed6d30b0e03c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.218479 May 23 16:11:27.543386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1426/2744] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:27 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:27.546857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-779cab02-c876-4c11-a3a2-3386cc6474de tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.216088 May 23 16:11:27.548144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1319/2745] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:27.548560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:27.549229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:27.555186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.555672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.556392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.556821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.559255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:27.559860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:27.566540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.566990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.567785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:27.568219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:27.629418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.637484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-864ae243-ba19-4b9a-920a-27ce9e850c48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093155 May 23 16:11:27.638056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1427/2746] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:27.709276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.717570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.719080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:27.720265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-74d410c1-9b07-46cc-b7cc-183ddfe259ba tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.164841 May 23 16:11:27.720825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1320/2747] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:28.560665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:28.561317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:28.564197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:28.564607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:28.565226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:28.568945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.572125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.572125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.572125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.572530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.572530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.572878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.573311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.754681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.762727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ef4fca4e-e270-4257-aee5-383d19781cf3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.206058 May 23 16:11:28.763646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1428/2748] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:28 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:28.766581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.768465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:28.769071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:28.774870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.775019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8a0018d1-b751-4bc5-90f1-7a13e83238c8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.214789 May 23 16:11:28.775385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.775831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1321/2749] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:28.776156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.776583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.780496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:28.781132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:28.787497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.787944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.788717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:28.789148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:28.846805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.854654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24ea2235-2c7d-48b2-ac26-0992db01e3f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090065 May 23 16:11:28.855243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1429/2750] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:28 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:28.930240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.939008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.940179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:28.941333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60f6b3ea-0b13-459c-86ba-4d5e81eb7ab1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.164458 May 23 16:11:28.941893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1322/2751] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:29.780371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:29.781002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:29.788443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:29.789228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:29.790972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:29.791418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:29.794178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:29.795047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:29.799507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:29.806670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:29.807555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:29.808763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:29.809449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:29.988470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:29.995143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:29.996337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f9a3218c-4503-4611-b7cb-eef2e55542e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220150 May 23 16:11:29.996914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1430/2752] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:30.002366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:30.002977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:30.003994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a474d472-012e-4786-91b8-210ea507a874 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.214694 May 23 16:11:30.008373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1323/2753] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:30.009241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:30.009848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:30.011681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:30.012130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:30.012966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:30.013347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:30.015696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:30.016202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:30.016961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:30.017666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:30.081818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:30.085485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:30.093318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-61d15823-3cb8-4bb8-b461-a4e2ab300c4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.095344 May 23 16:11:30.093914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1431/2754] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:30.163576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:30.172031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:30.174211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:30.175415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6bd2cfc1-1961-40d5-923b-0a3f66f88f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.169927 May 23 16:11:30.175975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1324/2755] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:31.013658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:31.014272 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:31.021765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:31.022235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.022369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:31.022478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.023191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.023626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.028687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.029133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.029993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.030429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.215257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.220348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.223946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b6f8feeb-0729-4386-b10f-39a286fc0275 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214127 May 23 16:11:31.224518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1432/2756] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:31.229672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61edeb86-54db-432e-8909-30feb38433ae tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210469 May 23 16:11:31.229942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:31.229942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:31.235660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1325/2757] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:31.240893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.241339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.242099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.242970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.244238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:31.244857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:31.250774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.251290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.252036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:31.252472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:31.330068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.338145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e2ab6fb6-eae4-40f2-b1fa-b696026d0917 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.112682 May 23 16:11:31.338916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1433/2758] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:31.395466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.398808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:31.404027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.405526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:31.409998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e915cc8-3d56-4743-8acd-a4a4288f5428 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170141 May 23 16:11:31.409998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1326/2759] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:32.242620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:32.243296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:32.246364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:32.246984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:32.250034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.250556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.251895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.252350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.255510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.255962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.256775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.257205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.440155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.448943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.449709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ff7b97e-050d-45ef-a73e-9f90074a3707 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211498 May 23 16:11:32.450541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1434/2760] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:32.455521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:32.456130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:32.457327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4ca22559-4928-439d-b91b-7da9058938a8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.214760 May 23 16:11:32.463382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.463833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.464590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.465020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.467183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1327/2761] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:32.472504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:32.473123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:32.479248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.479868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.480975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:32.481590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:32.530598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.540560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f7ed147-385f-4350-96a1-c37a67a135d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089121 May 23 16:11:32.550903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1435/2762] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:32.626032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.634975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.636490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:32.637646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eca92c6d-fb23-4377-bf37-ef4b83500f65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.169407 May 23 16:11:32.638194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1328/2763] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:33.466722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:33.467363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:33.474136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:33.474584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.474757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:33.474963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.475751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.476183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.481236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.481689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.482454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.482889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.675593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.678546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.683703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b475a7a3-1403-479e-811d-f568392941f5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213838 May 23 16:11:33.684378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1329/2764] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:33 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:33.686672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd162ef1-f0d1-40a6-ab1b-ffebbbfaa4f9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.223762 May 23 16:11:33.687405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1436/2765] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:33.689019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:33.689716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:33.691645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:33.692252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:33.697180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.697780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.697987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.698442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.698750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.699206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:33.699314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.699639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:33.785936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.793753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c83f5fde-1941-41cf-9f98-67f8b6269b74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.108363 May 23 16:11:33.794356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1330/2766] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:33.852447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.861499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.862993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:33.864167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1d48f37c-a06a-4e13-aa84-3e5d4121c570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.176027 May 23 16:11:33.864723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1437/2767] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:34.708553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:34.709176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:34.709829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:34.710447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:34.715722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:34.716174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:34.716499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:34.716940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:34.717059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:34.717371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:34.717680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:34.718107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:35.154967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.159575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.163778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c0ba8c28-61de-4f1f-a804-b93a5c777b6c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.459057 May 23 16:11:35.177582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1331/2768] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:34 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:35.177727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89688522-4091-4b7d-94cf-0bf7a0a8ad82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.461784 May 23 16:11:35.177727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1438/2769] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:35.177926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:35.177926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:35.178121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:35.178121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:35.178290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:35.178290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:35.178290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:35.178290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:35.179831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:35.180309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:35.181109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:35.181505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:35.258879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.270858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a58d85f-326d-442a-96f8-278f46e49f73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.105499 May 23 16:11:35.271744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1332/2770] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:35.331670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.340306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.341815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:35.343580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3ac555fc-7042-45ac-9011-5ee755cfbb72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.173550 May 23 16:11:35.344145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1439/2771] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:36.180774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:36.181387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:36.185109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:36.185881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:36.187725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.188672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.189706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.190145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.192445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.192891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.193639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.194070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.370938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.379583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75b48af9-82ad-4311-8e9d-246d282fc7fd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.202576 May 23 16:11:36.380151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1333/2772] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:36.384566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:36.385173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:36.385426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.391764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.392218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.393269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.393707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.394575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5fc4e820-49dd-43d9-86d9-4355e112baf2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213579 May 23 16:11:36.395161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1440/2773] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:36.399716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:36.400316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:36.405904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.406346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.407114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:36.407551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:36.459581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.468138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-97450f46-97c1-4a19-9039-73f8c672e392 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087006 May 23 16:11:36.468717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1334/2774] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:36.551188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.559537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.561043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:36.562204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6c26c20d-2f12-40e1-a65c-9736c9618db8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.166140 May 23 16:11:36.562762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1441/2775] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:37.396449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:37.397108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:37.403623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.404079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.404846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.405288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.411575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:37.412192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:37.418719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.419197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.419957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.420391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.589756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.598294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84d480f6-400b-4394-9e5a-5a9c0c8932d8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.205923 May 23 16:11:37.598871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1335/2776] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:37.603249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:37.603859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:37.609565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.610604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.611382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.611822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.613083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.620974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c9feb6a6-2a69-424f-a468-66ef682b79af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213188 May 23 16:11:37.621538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1442/2777] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:37.626036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:37.626640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:37.632213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.632656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.633405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:37.633838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:37.675671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.683740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd25b902-6c3a-444b-891f-3eef9d18ff6a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083866 May 23 16:11:37.684323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1336/2778] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:37 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:37.769080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.777442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.778934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:37.780134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-43fc912f-d59c-4a4b-a7b5-505f621ab42a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.157685 May 23 16:11:37.780686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1443/2779] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:38.003047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:11:38.003254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:11:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:11:38.615804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:38.616465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:38.622876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.623369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.624126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.624559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.637737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:38.638344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:38.644964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.645406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.646148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.646576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.817186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:38.820560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:38.825435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-93c3f543-4bef-4138-83b4-a0921b434426 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213554 May 23 16:11:38.825993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1337/2780] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:38.828938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f59dd513-9342-4bc2-98c7-fbe6095d770b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.195013 May 23 16:11:38.829620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1444/2781] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:38.830543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:38.831163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:38.841374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:38.841374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:38.841678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.841678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.841678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.841678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.842026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.842026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.842026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:38.842026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:38.909715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:38.917925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a760ddb-f415-4fbe-8bb6-a06f95477924 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091000 May 23 16:11:38.918479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1338/2782] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:38.989257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:38.997456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:38.998939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:39.000116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a760fe94-6fef-46cf-875d-4a0c2a1ecd95 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.169741 May 23 16:11:39.001299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1445/2783] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:39.841759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:39.842379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:39.848502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:39.848969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:39.849108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:39.849346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:39.850113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:39.850549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:39.855456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:39.855904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:39.856647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:39.857076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:40.043168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.050139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.051712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ad7e0672-4b80-4680-af70-bc8d16f80ec2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213807 May 23 16:11:40.052287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1339/2784] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:40.056722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:40.057336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:40.058784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76486da8-51bc-4837-bec6-9b4718516e6c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213984 May 23 16:11:40.059408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1446/2785] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:40.063637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:40.064097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:40.064301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:40.064857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:40.064988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:40.065293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:40.070982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:40.071533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:40.072423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:40.072923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:40.147607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.155421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c109340-0a95-417b-8714-36a96a52ab25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.102161 May 23 16:11:40.155988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1340/2786] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:40.212079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.220653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.222171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:40.223377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aae7d379-f435-444a-ad06-78d7e28aecd9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.163170 May 23 16:11:40.223967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1447/2787] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:41.068068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:41.068693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:41.075245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.075700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.076470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.076909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.077591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:41.078216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:41.085088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.085533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.086281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.086712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.104900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:41.270006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.273216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.279360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4404665-13b8-41fc-ab6f-0927177656eb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215011 May 23 16:11:41.280878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1341/2788] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:41 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:41.287394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:41.288083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:41.288242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77b26d92-4cfa-404c-a995-d3a0f41ed53a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.214290 May 23 16:11:41.288990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1448/2789] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:41.294444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.294888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.295671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.296105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.302827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:41.303461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:41.309500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.309950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.310720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:41.313580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:41.363058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.371145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-26a4851f-864e-4efa-bcd9-e925745c2660 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087991 May 23 16:11:41.371715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1342/2790] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:41.467507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.476936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.478651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:41.479641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8956518c-27d3-4993-951f-b69202b57f68 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.180615 May 23 16:11:41.480320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1449/2791] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:42.299396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:42.300019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:42.305336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:42.305946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:42.306510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.306962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.307750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.308187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.312333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.312813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.313572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.314005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.326272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:42.499980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.504458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.508170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c054d1d6-42e9-4c58-957a-6f57c9f461c5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.206542 May 23 16:11:42.508746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1450/2792] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:42.512704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-850a8059-e7f4-4afc-9a54-99594bc4b2c1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217780 May 23 16:11:42.513082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:42.513519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1343/2793] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:42.518488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:42.518890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:42.519118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:42.525074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.525527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.526290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.526728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.528055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.528559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.529370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:42.529861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:42.615613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.624183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-643cd07f-361d-4a2a-ac0d-1dd1b3d5715b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.114484 May 23 16:11:42.624765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1451/2794] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:42.679739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.688027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.689525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:42.690681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-10795f79-827d-44f2-a015-31cc1c246605 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.176381 May 23 16:11:42.691265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1344/2795] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:43.525923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:43.526546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:43.529446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:43.530071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:43.533846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.534305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.535066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.535537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.546697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.546697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.546697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.546697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.727088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.731250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.735654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9f00d0c5-eb2e-44a6-952f-79cbee053185 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213212 May 23 16:11:43.736227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1452/2796] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:43.739742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb11971b-2302-460c-ada4-db62aa94c7a5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214055 May 23 16:11:43.740461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1345/2797] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:43.741180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:43.741808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:43.747774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.748242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.748988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.749430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.753220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:43.753834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:43.761373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.761373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.761373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:43.761373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:43.824060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.832055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e356b16-dc35-4a63-b3c2-f0f7b2bafa22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.094886 May 23 16:11:43.832626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1453/2798] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:43.907569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.915978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.917522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:43.918677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4abc08c4-3105-4f91-a959-aa96e81aeae2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.177487 May 23 16:11:43.919243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1346/2799] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:44.755735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:44.756339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:44.756924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:44.757533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:44.762820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.763303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.763872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.764065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.764328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.764499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.765072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.765502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.958160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:44.961508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:44.966324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af79427d-413b-43f3-b45d-af86cc5b19db tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214391 May 23 16:11:44.966900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1454/2800] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:44.971886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:44.972492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:44.977491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b6c2388f-53f9-4877-92fe-40a8c3df05fe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.224200 May 23 16:11:44.978064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1347/2801] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:44.978567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.979011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.979809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.980248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.982471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:44.983085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:44.989162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.989684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:44.990509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:44.991011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:45.066511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:45.074440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a8e0256-fe82-4080-a0aa-ac4e680ba902 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106561 May 23 16:11:45.075019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1455/2802] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:45.132653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:45.141777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:45.143569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:45.144725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f70932b3-b641-49d5-8ce7-3c88cea66d4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.165760 May 23 16:11:45.145273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1348/2803] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:45.983947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:45.984699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:45.991254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:45.995254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:45.995254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:45.995254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:45.995602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:45.995602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:46.001567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.002015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.002775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.003247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.182278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.189308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.191772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5b39ab1-3048-4eae-838a-d7f77b725fb4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211300 May 23 16:11:46.192254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1456/2804] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:46.196664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:46.197269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:46.198122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ea25bf72-c93b-4fbe-b7d2-ba85b8280868 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.207707 May 23 16:11:46.198679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1349/2805] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:45 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:46.203421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.203886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.204049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:46.204647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.205079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.205496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:46.214195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.214938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.216350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:46.217145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:46.272386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.280522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-98832e29-edb7-48a4-a70b-d14c5336bd18 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087263 May 23 16:11:46.281103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1457/2806] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:46.363320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.371767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.373306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:46.374473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b11e702-8d66-4f05-b28f-272470aec151 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.174563 May 23 16:11:46.375025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1350/2807] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:47.211830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:47.212783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:47.214208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:47.214831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:47.222794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.223064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.223286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.223436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.224063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.224170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.224507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.224653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.419128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.426268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.427273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae78af28-4641-4917-be46-a5868881358f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.216829 May 23 16:11:47.427847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1351/2808] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:47.432247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:47.433066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:47.434556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-849ec87e-3f67-4153-aec0-d1f012e3054c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.228955 May 23 16:11:47.435140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1458/2809] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:47.440350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:47.440965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:47.443461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.443643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.444688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.445133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.446820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.447318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.448114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:47.448555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:47.532179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.540203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a6427f4b-74d1-47ab-9fb3-5765d7a11064 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.111392 May 23 16:11:47.540775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1352/2810] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:47.599313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.607718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.609236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:47.611011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7dbfca33-8309-4709-ad9a-f98463eb90db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.174666 May 23 16:11:47.611602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1459/2811] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:48.444293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:48.444838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:48.451390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.451558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:48.451954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.452054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:48.452574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.453001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.458911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.459425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.460180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.460610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.648915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.652411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.656872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-13f0de7a-6bbd-44b4-a631-93eb6ea67640 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.216443 May 23 16:11:48.657439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1353/2812] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:48.660446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5b343918-1194-4680-89eb-1725ffcda80f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212759 May 23 16:11:48.661148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1460/2813] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:48 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:48.665954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:48.666567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:48.670593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:48.671308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:48.672867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.673328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.674085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.674521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.677567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.678018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.678775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:48.679231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:48.760130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.768163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2884fd0b-021e-4c15-be22-953a2deec509 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.109513 May 23 16:11:48.768729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1354/2814] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:48.825426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.833922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.835440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:48.836589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9a644659-1ec7-4d01-8315-5fa3ac082961 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.174133 May 23 16:11:48.837140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1461/2815] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:49.673801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:49.674451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:49.677390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:49.678012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:49.680979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.681430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.682186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.682617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.684441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.684930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.685646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.686085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.877458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:49.880506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:49.885836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b4f07993-387a-41bd-98b3-7c660f32b0ac tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.215861 May 23 16:11:49.895671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1355/2816] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:49.895876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76311487-d8d3-4fbf-b8cf-df9c4f0d7a63 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215199 May 23 16:11:49.895876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1462/2817] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:49.896103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:49.896103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:49.896292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:49.896292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:49.898682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.899171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.899941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.900376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.900950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.901405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.902160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:49.902589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:49.986431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:49.994368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d8376746-b951-4bf3-859b-d06d3b8ba21b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106449 May 23 16:11:49.994978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1356/2818] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:50.052539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:50.060811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:50.062295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:50.063482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7da759b-17f0-4b8a-9f31-0293c6884225 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.173268 May 23 16:11:50.064042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1463/2819] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:50.904153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:50.904782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:50.905820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:50.906470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:50.911118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:50.911616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:50.912414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:50.912856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:50.912971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:50.913341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:50.914084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:50.914521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:51.103883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.107215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.112325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c081384d-6f39-43c1-8ba3-7d69998a5fbe tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212819 May 23 16:11:51.112893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1357/2820] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:51.115749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ed47bdc2-bb07-4c26-ba38-614f7c294934 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213735 May 23 16:11:51.116487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1464/2821] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:51.120714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:51.121313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:51.123274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:51.123909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:51.127166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:51.127616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:51.128368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:51.128801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:51.129781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:51.130236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:51.131003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:51.131485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:51.132670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:51.213856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.222801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1dd92a0a-b4bc-44b6-97e8-99b442bbdc73 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.108900 May 23 16:11:51.223410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1358/2822] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:51.277399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.286317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.287851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:51.289014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-063aad85-3985-47d8-a205-0b55be603705 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.171784 May 23 16:11:51.289572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1465/2823] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:52.129956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:52.130502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:52.131819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:52.132440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:52.137312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.137760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.138513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.138945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.139057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.139432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.140300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.141471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.328617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.335478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.336651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-797d38c1-aa21-421e-a341-65025bf8b8f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210581 May 23 16:11:52.337507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1359/2824] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:52.341889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:52.342502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:52.343554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-883c4fb0-fc27-4fb1-995b-4ef4d5073884 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215451 May 23 16:11:52.345574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1466/2825] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:52.348467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:52.349078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:52.349191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.349653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.350485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.350858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.354739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.354911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:52.355455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.355979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:52.356417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:52.423734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.431568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1eb7b340-2f59-4685-87c6-91bf543b7c23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093225 May 23 16:11:52.432132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1360/2826] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:52.504720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.513021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.514526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:52.515728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dde0f2fc-cbdd-4174-958d-f76c44d00f6d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170706 May 23 16:11:52.516288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1467/2827] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:53.353360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:53.353982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:53.360798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:53.361264 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.361543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:53.361666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.362316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.362763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.367974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.368425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.369187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.369626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.565235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.569137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.573409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cbdb649e-d2d0-4073-ae43-55961f48570c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.223827 May 23 16:11:53.575619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1361/2828] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:53.577486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b01cbf1-92e7-4be8-87c5-0e0e6dfe1699 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220996 May 23 16:11:53.578194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1468/2829] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:53.578876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:53.584056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:53.585590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:53.585792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.586208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:53.586338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.587028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.587556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.592365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.595083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.595083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:53.595083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:53.671242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.679756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8b0750d5-cea0-45e2-af2e-a2e8d9ddfc8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.104827 May 23 16:11:53.680622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1362/2830] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:53.737391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.745610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.747127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:53.748287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2274bf5-d6e9-4212-85d7-099a004c4b92 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.169285 May 23 16:11:53.748840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1469/2831] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:54.591956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:54.592622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:54.593608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:54.594833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:54.599712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.600162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.600917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.601347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.601473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.601795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.602544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.602976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.792653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.795716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.800929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52db2539-5c9c-41ba-9138-04bcad16a721 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213046 May 23 16:11:54.801503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1363/2832] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:54 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:54.803656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f33ec956-92a4-4f28-b314-40017f079deb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213830 May 23 16:11:54.804388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1470/2833] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:54 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:54.808623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:54.809233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:54.815387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:54.815586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.816012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:54.816148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.816794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.817233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.821931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.822383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.823179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:54.823630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:54.908431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.925341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b7c59bf-a70d-4511-98cf-9128bbc86baa tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.122784 May 23 16:11:54.925934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1364/2834] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:54.974168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.982904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.984450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:54.985609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e7c6ea01-d828-43a4-bae1-da3c7fa63b78 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.180488 May 23 16:11:54.986173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1471/2835] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:55.818228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:55.818843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:55.820415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:55.821034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:55.825095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:55.825536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:55.826290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:55.826717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:55.827499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:55.827954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:55.828744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:55.829180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:55.970169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:11:55.970371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:11:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c (ip 10.222.0.32) !!! May 23 16:11:56.016609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.019986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.025367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-53fa7842-ea93-4d8c-b913-6951bd2273a1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210962 May 23 16:11:56.025931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1365/2836] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:56.027814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32866f0c-b5d0-4670-95c7-ae613e1be2f8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.211263 May 23 16:11:56.028520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1472/2837] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:56.030390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:56.030993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:56.033298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:56.033906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:56.037456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:56.037903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:56.038657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:56.039088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:56.039565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:56.040012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:56.040763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:56.041194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:56.122242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.130734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1da4d8e2-82c4-44b6-8b17-85508a88a729 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.103827 May 23 16:11:56.131338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1366/2838] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:56.191076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.199773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.201280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:56.202447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32bc4458-521e-4538-93bc-2335b1402f1e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.173187 May 23 16:11:56.203002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1473/2839] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:57.052034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:57.052622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:57.053016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:57.053120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:57.059159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.059362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.059613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.059748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.060368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.060484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.060799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.060924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.248968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.253310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.256896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-85a32717-a685-479e-a6bd-e06d1570fcf6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208450 May 23 16:11:57.257465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1474/2840] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:57.261482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e9d8103-695a-48c4-8da8-f1d6db039927 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213270 May 23 16:11:57.261861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:57.270010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1367/2841] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:57.270173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:57.270268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:57.270268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:57.270468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.270468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.270468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.270468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.273458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.273938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.274711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:57.275209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:57.342019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.350525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-112c8578-7899-418f-a87c-71275566cad1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092119 May 23 16:11:57.351108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1475/2842] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:57.421762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.430678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.432211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:57.433369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95f6a2d9-607d-4526-b6b0-ee4054acc852 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170286 May 23 16:11:57.433924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1368/2843] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:58.274169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:58.274798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:58.278913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:58.279552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:58.281331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.281786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.282557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.283001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.285937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.286386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.287158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.287598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.477395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.484279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.485351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-401222ec-4f10-4a90-9965-50a9a4b2947c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215154 May 23 16:11:58.486109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1476/2844] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:58.490305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:58.490917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:58.495569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-363b3c30-ecfb-4977-8c96-91914941ecfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.220687 May 23 16:11:58.496142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1369/2845] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:58.500692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:58.501330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:58.501769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.502236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.503018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.503490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.507186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.507646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.508414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:58.508859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:58.586535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.594481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4a54e79-f614-4389-acb1-7f9e8ecda7f1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107616 May 23 16:11:58.595051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1477/2846] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:58.655703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.664131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.665646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:58.666820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ad15eece-9874-460e-9b04-243c8a2c6fec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.169755 May 23 16:11:58.667424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1370/2847] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:11:58.788833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:11:58.789024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:11:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:11:59.501799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:59.502442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:59.504745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:59.509636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.510082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.510841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.511309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.516633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:59.516633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:59.520025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.520480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.521267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.521686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.707705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.710871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.715820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bfab0113-66c7-4f6e-a2a8-d9fe4b03b5ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217913 May 23 16:11:59.716477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1478/2848] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:59.718733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-42f8959a-bc87-453b-b802-6570ed0d3a71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210223 May 23 16:11:59.726143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1371/2849] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:11:59.726303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:59.726303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:59.726500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:11:59.726500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:11:59.727582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.728035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.728804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.729239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.730124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.730581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.731368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:11:59.731814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:11:59.816187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.824290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-50baf265-6c04-49f9-8be2-7f6cecac4e14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106692 May 23 16:11:59.825241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1479/2850] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:11:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:11:59.883566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.892070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.893843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:11:59.895011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8ad33cc2-e829-4b6a-b80b-10b416eb1fa7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.174778 May 23 16:11:59.895606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1372/2851] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:11:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:00.732637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:00.733260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:00.735391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:00.736015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:00.738384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:00.739820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.740283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.741036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.741466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.743000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.743482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.744244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.744680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.930341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:00.933747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:00.939389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d293ed4-b4b9-434d-8ea4-733e8787c4d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210557 May 23 16:12:00.939968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1480/2852] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:00 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:00.942315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8aa12a4d-ac3c-41cd-9118-a02d0d06f309 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210670 May 23 16:12:00.943037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1373/2853] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:00 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:00.944394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:00.945006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:00.947553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:00.948029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:00.950869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.951444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.952233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.952728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.954211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.954656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:00.955511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:00.955945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:01.026483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:01.029926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:01.034775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2fb223dc-43fd-49ec-81b2-496fdb295e17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093868 May 23 16:12:01.035371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1481/2854] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:01.109574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:01.117849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:01.119408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:01.120556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c994d21c-349e-496a-871a-267f3a30bb94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.176706 May 23 16:12:01.121133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1374/2855] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:01.960049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:01.960670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:01.967584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:01.968044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:01.969604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:01.969912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.108308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.116556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5fa46651-b4fa-4350-a9fb-b971318ded64 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160739 May 23 16:12:02.117121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1482/2856] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:02.121527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:02.122056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:02.122250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:02.122342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:02.128213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.128555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.128675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.129007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.129411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.129771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.129875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.130208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.217377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.226039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-768ea569-edb6-42fc-9b58-c82376317e84 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107885 May 23 16:12:02.226620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1483/2857] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:02.233747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:02.233747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:02.239675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.240192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.240974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:02.241415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:02.293531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.301581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e431cfd5-22cd-45af-9fd0-49c8be7d99c5 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.339279 May 23 16:12:02.302150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1375/2858] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:02.376852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.385787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.387319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:02.388473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f6b5886d-e418-4cb3-bbbd-5332e675da84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.160886 May 23 16:12:02.389029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1484/2859] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:03.133500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:03.134108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:03.140813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.141270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.142031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.142472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.243287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:03.244148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:03.250884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.251355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.252116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.252550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.273537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.279847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:03.284276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aeceedde-ff09-4cde-905e-e1e0491e2997 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154628 May 23 16:12:03.284907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1376/2860] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:03.318896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.319612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:03.320396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:03.327544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.327714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2b783312-f687-4dc2-8721-94261334d5ef tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088307 May 23 16:12:03.328939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1485/2861] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:03.329086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.329702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.330408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.408668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:03.409189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:03.415339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.415792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.416604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:03.417044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:03.473946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.488985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b759c46e-8912-41b9-ae5c-69edb8d47b2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.174260 May 23 16:12:03.489864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1377/2862] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:03.559599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.568924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.570431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:03.571642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2f71f9bf-2740-413d-9c23-62a995a40d66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.167583 May 23 16:12:03.572211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1486/2863] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:04.301972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:04.302590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:04.309324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.309771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.310520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.310950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.345907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:04.346624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:04.353322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.353771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.354527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.354962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.426425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.434219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08bb4de8-7ae5-40e4-8edf-5397c75ecd5c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092845 May 23 16:12:04.434993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1487/2864] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:04.454588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.462375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39f5aad7-e4ee-434f-9510-191ae4f5e73a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.164662 May 23 16:12:04.462931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1378/2865] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:04.507382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:04.508004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:04.514046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.514493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.515277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.515741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.590010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:04.590633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:04.597244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.597769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.598596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:04.599120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:04.654260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.662579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-edbaa40e-1f93-4006-bdcc-491a8e83d635 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.159109 May 23 16:12:04.663136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1488/2866] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:04.735498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.743760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.745245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:04.746396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3e655edf-5723-4dfb-995b-bb7d84a0d905 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.160255 May 23 16:12:04.746953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1379/2867] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:05.450980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:05.451678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:05.458216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.458660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.459426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.459904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.479770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:05.480501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:05.489430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.490751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.491693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.492131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.526811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.535319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8cbee85d-beb3-473a-abf9-24a4052cbcb7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088099 May 23 16:12:05.535979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1489/2868] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:05 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:05.627897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.635832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ebb8bb67-1f2f-47e8-8418-edd569aadda9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159845 May 23 16:12:05.636406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1380/2869] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:05.679482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:05.680130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:05.686086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.686534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.687331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.687777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.764744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:05.765352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:05.772055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.772499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.773245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:05.773673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:05.822507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.831048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0653b6b-a49f-4343-9c5c-0c0cbe6ccb43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.155458 May 23 16:12:05.831643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1490/2870] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:05.902986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.911855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.913344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:05.914493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8859f6a0-b897-4750-83a6-7c2311164d74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.153773 May 23 16:12:05.915044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1381/2871] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:06.552891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:06.553507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:06.559638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.560088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.560840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.561277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.653917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:06.654539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:06.661326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.661772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.662541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.663005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.738054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:12:06.738324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:12:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c (ip 10.222.0.32) !!! May 23 16:12:06.867923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:06.876078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24c45e6f-05bb-4d02-b732-bea8251557f5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.327013 May 23 16:12:06.876876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1491/2872] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:06.881217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:06.881824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:06.887525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.887965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.888710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:06.889141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:06.996963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.005217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-87e26a58-9968-4a29-b159-00ff9ae22376 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.355225 May 23 16:12:07.005884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1382/2873] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:07.010459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:07.011166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:07.017272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:07.017979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:07.018762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:07.019231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:07.042553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.051117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4af9f0a-33dc-4393-98a1-ce64efe933e1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.173359 May 23 16:12:07.051707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1492/2874] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:06 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:07.145260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.153565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.155089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.156268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdd0fca4-75d9-4b31-9d4a-46dc9ac55807 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.149392 May 23 16:12:07.156825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1383/2875] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:07.893806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:07.894446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:07.900968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:07.901416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:07.902164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:07.902596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:07.961819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:07.969627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-326a8eba-278b-460e-8a47-e8f0bc6774b5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080542 May 23 16:12:07.970188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1493/2876] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:08.022373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:08.022993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:08.029433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.029884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.030642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.031129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.068522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:08.069165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:08.075202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.075656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.076428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.076847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.214082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:08.214586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72f4c4d0-34f2-484c-9967-e7628a08c127 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.195974 May 23 16:12:08.224314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1384/2877] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:08.224314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:08.224314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:08.226341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.226792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.227590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.228027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.236334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:08.244200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7ea3d42d-8b2b-4fd1-aeb8-35b0a24ffe27 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.179532 May 23 16:12:08.244779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1494/2878] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:08 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:08.361691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:08.370010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:08.371556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:08.372713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-437e60a9-a7ce-4766-bd47-9b0b49fa5af9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.156615 May 23 16:12:08.373270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1385/2879] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:08.985741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:08.986354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:08.992947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.993396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:08.994177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:08.994612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.054910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.063426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a074bdcd-5879-4e55-a3c5-e0446d21a6df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081481 May 23 16:12:09.063988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1495/2880] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:09.230922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:09.231549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:09.238866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.239293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.240078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.240500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.261612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:09.262232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:09.268521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.268970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.269728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.270164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.429672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.437711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-118f682f-8f53-48b6-9dc1-dbaef7f080e2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.210650 May 23 16:12:09.438278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1386/2881] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:09.442718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:09.443425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:09.446445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.449142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.449595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.450356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:09.450789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:09.454842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eb45bfdc-0f3a-44d1-bd9c-f693cf294cdf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.197096 May 23 16:12:09.455448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1496/2882] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:09.580117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.588594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.590300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:09.591504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1e256a73-91dd-4c3d-84e2-7ae24bdefe42 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.152280 May 23 16:12:09.592052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1387/2883] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:10.079969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:10.080586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:10.087184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.087639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.088389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.088825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.148900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.156812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5412e87a-6ba1-49c5-89a8-3e45ddf076ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081387 May 23 16:12:10.157373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1497/2884] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:10.453947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:10.454474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:10.461223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.461670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.462423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.462855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.471612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:10.472233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:10.478150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.478597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.479376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.479820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.497830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:10.661072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.663997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.669003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-180c2b37-c382-49aa-9a9a-93621b47e225 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.218967 May 23 16:12:10.669580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1388/2885] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:10 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:10.672488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c5b593d9-381a-4851-a4f3-e1c75387c16d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.204115 May 23 16:12:10.673203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1498/2886] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:10.674489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:10.675115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:10.680809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.681262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.682026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:10.682465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:10.812932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.821290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.822784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:10.823969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d0a8a0b2-43a4-4dcd-9cf7-0331782ef338 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.153487 May 23 16:12:10.824527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1389/2887] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:11.174569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:11.175122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:11.181558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.182003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.182754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.183210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.245538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:11.254084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-976dd75c-0137-4810-ac16-bf07a46aef14 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084296 May 23 16:12:11.254644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1499/2888] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:11.686633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:11.687305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:11.688922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:11.689535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:11.693900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.694353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.695139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.695588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.695791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.696234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.696981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.697412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.888618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:11.892234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:11.897737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9070490c-5802-4d38-a62c-95b8529dd479 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.214870 May 23 16:12:11.898507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1390/2889] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:11.901003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c512b1ca-b08b-40e0-b63e-3490ec3bf32d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215977 May 23 16:12:11.911039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1500/2890] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:11.911238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:11.911238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:11.911238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.911238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:11.911654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:11.911783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.037959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:12.046374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:12.047913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:12.049073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-913ecc9b-42fa-4d3d-bb54-f6d35ffc6cdd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.149853 May 23 16:12:12.049630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1391/2891] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:12.270943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:12.271512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:12.277948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.278390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.279158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.279595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.342504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:12.350941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb376b3b-dbb7-4431-8edf-667532d4b0c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084176 May 23 16:12:12.351529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1501/2892] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:12.916131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:12.916738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:12.917538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:12.918169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:12.923337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.923790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.924126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.924579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.924694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.925024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:12.925335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:12.925772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:13.116435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.120522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.124441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-484ba03b-e684-4dac-8ab8-7874583fac68 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212181 May 23 16:12:13.125531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1392/2893] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:12 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:13.128917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8d86374e-1844-4d54-ab18-c9838d9a5702 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.215170 May 23 16:12:13.129647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1502/2894] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:13.129958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:13.130604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:13.136414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:13.136870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:13.137635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:13.138077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:13.156229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:13.264040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.272632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.274146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.275353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2b28d7ab-46ff-4a38-8ee4-eb02a9288f25 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.148866 May 23 16:12:13.275917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1393/2895] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:13.368400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:13.369020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:13.376102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:13.376554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:13.377314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:13.377752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:13.438020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:13.445976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c22e66a5-0c6c-44db-b3d0-b84df3928737 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081394 May 23 16:12:13.446539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1503/2896] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:14.142310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:14.143311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:14.147277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:14.148399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:14.150196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.150655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.151466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.151903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.156289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.156735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.157495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.157924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.348359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.351765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.356468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-95399129-aa8a-496a-8e73-d808fb19ec18 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.218053 May 23 16:12:14.357002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1394/2897] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:14.361020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-658e9d0b-5649-47b8-b995-2fc778b17b13 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219255 May 23 16:12:14.363138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1504/2898] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:14 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:14.363293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:14.363293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:14.368665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.369153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.369907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.370346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.462845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:14.463475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:14.470500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.470960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.471757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:14.472196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:14.500807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.509460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.511005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.512221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7cd3b4a2-8991-4f5d-8fa3-f7aaf15da81e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.154304 May 23 16:12:14.512798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1395/2899] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:14.533021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:14.541048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-72e4493d-389a-4b3c-93e7-0a0f1eb1a14b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082203 May 23 16:12:14.541613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1505/2900] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:15.373986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:15.374606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:15.377395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:15.378006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:15.381150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.381602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.382361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.382800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.383979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.384424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.385170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.385598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.582206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.586485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.590500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25e37a7c-6230-4c75-8b57-c2df34db52f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.220466 May 23 16:12:15.591079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1396/2901] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:15.595294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-675ab19f-2fd2-4dc4-a362-d056952986d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.221509 May 23 16:12:15.604583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1506/2902] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:15.604783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:15.604783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:15.605035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:15.605035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:15.605234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.605234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.605234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.605234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.608318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.608760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.609572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:15.610005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:15.698485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.707151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f7b2a9a2-5855-4df2-8121-13f4789f310b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.110208 May 23 16:12:15.707721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1507/2903] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:15.759067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.768087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.769586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:15.770739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-261a8143-156f-44cd-8a91-a3c6fe079a7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.178738 May 23 16:12:15.771381 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1397/2904] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:16.608333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:16.608985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:16.613305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:16.613948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:16.616084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.616531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.617283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.617714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.620392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.620839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.621599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.622032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.809062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.812341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.817525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e7443d03-2f65-4626-a14d-5135e80b49b3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212939 May 23 16:12:16.818368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1508/2905] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:16 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:16.820471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8eacd3fb-4d19-411c-a9ad-90fbcd4d08f1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212306 May 23 16:12:16.821183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1398/2906] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:16.827156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:16.827156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:16.827430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:16.827430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:16.829562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.830115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.830917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.831396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.832370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.832825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.833579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:16.834019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:16.905669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.914361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a9213084-0c40-4815-97ea-f4b1f6e7c8b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.095118 May 23 16:12:16.914942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1509/2907] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:16.987506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.995833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.997316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:16.998610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e4b6c00f-8962-4407-88f8-2cd0c7207ee9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.176610 May 23 16:12:16.999186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1399/2908] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:17.834535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:17.835159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:17.837514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:17.838135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:17.842138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:17.842590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:17.843412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:17.843862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:17.849147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:17.849595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:17.850368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:17.850796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:18.039586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.042450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.047623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e3c1a0c6-f0cd-4456-bd4a-37dc2dc4d814 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.217081 May 23 16:12:18.048199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1510/2909] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:18.057204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a215f83-fcda-48b0-b791-39c56334dd16 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216966 May 23 16:12:18.057204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1400/2910] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:17 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:18.057426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:18.057426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:18.057601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:18.057601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:18.059312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:18.059760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:18.060516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:18.060950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:18.063032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:18.063506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:18.064261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:18.064697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:18.129710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.137494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0968e2e0-2eb4-417e-9413-9ec5893fa2f0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088255 May 23 16:12:18.138055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1511/2911] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:18.214517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.223358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.224849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:18.226000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-992faec4-98cf-4f10-9de3-3309d7ea6830 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.173657 May 23 16:12:18.226553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1401/2912] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:19.065188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:19.065798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:19.066955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:19.067609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:19.072257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.072704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.073450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.073879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.073988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.074365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.075122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.075562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.269150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.272014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.277157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae2a3cb2-6120-4e9b-8dbf-2c5bef51c4c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213985 May 23 16:12:19.278021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1402/2913] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:19.279965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70a756e8-2da9-45dc-8594-0cd9e69b187d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.219093 May 23 16:12:19.286948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1512/2914] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:19.287156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:19.287156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:19.287370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:19.287370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:19.288915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.289373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.290154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.290596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.291297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.291743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.292490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:19.293109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:19.372365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.380790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9ae5aed-ff57-4409-8609-821461e71e66 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.101942 May 23 16:12:19.381418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1403/2915] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:19.438331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.446683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.448218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:19.449377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02232bc1-0ecc-45b4-92b9-d3ee677e3d53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.167978 May 23 16:12:19.449934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1513/2916] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:20.295487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:20.296104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:20.296816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:20.297514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:20.303330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.304079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.304235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.304645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.304978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.305448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.305580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.305887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.503904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.507448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.512877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d0ba112-c2e5-403e-81fa-277cf433c06b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.220007 May 23 16:12:20.513456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1514/2917] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:20 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:20.516337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dea5f4a5-9a0f-44e4-bf65-70c8ba3c9d10 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.226002 May 23 16:12:20.526814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1404/2918] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:20 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:20.527023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:20.527023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:20.527258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:20.527258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:20.527419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.527419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.527419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.527419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.529226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.529681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.530445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:20.530872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:20.530990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:20.614468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.622369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-79929319-8cbe-4403-97c3-da590b6ff4fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107971 May 23 16:12:20.622941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1515/2919] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:20.680644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.689699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.691241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:20.692410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-88dcae40-673d-4448-a1c3-fe18bef3dd7c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.173280 May 23 16:12:20.692966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1405/2920] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:21.530440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:21.531647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:21.534216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:21.534840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:21.538243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.538702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.539479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.539919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.541238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.541697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.542462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.542901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.730463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.733901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.738926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-286bf1f7-c108-45fa-b038-bdd00a3b2f9b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212405 May 23 16:12:21.748750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1516/2921] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:21.748950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-25ac7dec-f700-498f-b706-9abb2da52e7a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212296 May 23 16:12:21.748950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1406/2922] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:21 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:21.749135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:21.749135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:21.749300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:21.749300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:21.750454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.750914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.751704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.752163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.753461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.753913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.754673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:21.755128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:21.822158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.830726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3a1abfe4-671e-4427-ae5c-5fde45d384a6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090226 May 23 16:12:21.831510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1517/2923] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:21.903472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.912202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.913857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:21.915011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ea033e1b-9f30-458c-be26-0327a050c33b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.171605 May 23 16:12:21.915677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1407/2924] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:22.755663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:22.756310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:22.758995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:22.759676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:22.762834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.763329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.764094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.764527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.766078 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.766529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.767313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.767754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.956277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:22.959384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:22.964593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5dd05300-ab7a-4c38-b76a-208537f59cd8 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212789 May 23 16:12:22.965217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1518/2925] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:22.967693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-739520de-a8f5-43ae-8e71-6a168b7103ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.212570 May 23 16:12:22.969008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1408/2926] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:22.970392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:22.971051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:22.976968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.977411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.978165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.978591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.982357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:22.986654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:22.988850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.989306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:22.990089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:22.990530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:23.048500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:23.056400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02fa7762-6ab1-4929-abc5-d593a4b1fc52 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090230 May 23 16:12:23.056970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1519/2927] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:23.131985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:23.140920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:23.142419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:23.143867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fc4d0647-0c62-4dce-933c-059879ffe96b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.165207 May 23 16:12:23.144426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1409/2928] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:23.982863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:23.983513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:23.985053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:23.985703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:23.990558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:23.991002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:23.991779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:23.992209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:23.992328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:23.992844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:23.993622 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:23.994059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:23.998170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:24.184093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.187900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.192632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-223b59fe-7624-4c7d-9971-5d157819cd28 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.213744 May 23 16:12:24.193196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1520/2929] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:24.196148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-687d08d4-549b-48a7-aca0-783983c1db37 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.214939 May 23 16:12:24.196881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1410/2930] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:24.201114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:24.201725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:24.208155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:24.209198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:24.209331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:24.209443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:24.209852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:24.209988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:24.216163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:24.216617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:24.217409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:24.217844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:24.304240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.312099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2cc9f8d9-da27-4391-ab33-e8e3870b376c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.117957 May 23 16:12:24.312670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1521/2931] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:24.361834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.370240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.371839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:24.373005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd809606-6e18-4173-a9ec-01747f58bab6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.175389 May 23 16:12:24.373556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1411/2932] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:25.210391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:25.211235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:25.213917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:25.214626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:25.217836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.218279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.219061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.219520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.220731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.221168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.221926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.222364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.404861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.412988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d04f577c-9cd7-4b27-914a-b1c8ca6c27d3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.206865 May 23 16:12:25.413568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1522/2933] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:25 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:25.418043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:25.418656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:25.421005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.425235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.425695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.426464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.426902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.429839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1bca2452-e6e1-4ec0-9a05-5a115d9cc5c2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220298 May 23 16:12:25.430505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1412/2934] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:25.434951 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:25.435757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:25.442103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.442552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.443338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:25.443774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:25.493258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.502155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae9800ef-f085-4be6-9ec2-9754f6be91ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087532 May 23 16:12:25.502735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1523/2935] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:25.586563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.595780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.597281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:25.598554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c906750-2504-4868-a2b8-996aa7af2d02 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.167253 May 23 16:12:25.599115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1413/2936] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:26.429972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:26.430627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:26.437032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.437475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.438223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.438651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.447540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:26.448167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:26.454682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.455167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.455939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.456379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.629712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.637540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8402f10a-0b72-46e1-ac26-42d99ef07acb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.211640 May 23 16:12:26.638111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1524/2937] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:26 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:26.640886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.642488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:26.643417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:26.649089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.649535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.649659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2fc0542c-bd0b-47de-a1e1-54b68186439b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.205834 May 23 16:12:26.650099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1414/2938] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:26.650288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.650723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.654735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:26.655395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:26.661787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.662247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.663045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:26.663525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:26.713033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.721642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-53b76380-0937-4b79-a5f4-fc7a250736d0 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082545 May 23 16:12:26.722219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1525/2939] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:26 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:26.806177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.814601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.816129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:26.817279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-11cc26aa-878f-4f4d-902a-9c842940ae2b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.166276 May 23 16:12:26.817823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1415/2940] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:27.654688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:27.655316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:27.662498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.662986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.663778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.664216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.666909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:27.667543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:27.673956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.674406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.675183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.675619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.801321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:27.809941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1a9e306-7a7a-42dd-bda1-03c3d557348c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159090 May 23 16:12:27.810542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1526/2941] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:27.815824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:27.816543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:27.822721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.823206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.823983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.824425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.894815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:27.902702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ac243b4-3315-428a-a156-61968020a88a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091206 May 23 16:12:27.903290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1527/2942] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:27.907962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:27.908490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:27.910187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:27.914808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.915296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:27.916057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:27.916493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:28.019894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:28.027957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-76a3f293-a5b3-44a8-83bc-a1a3e4486800 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.364834 May 23 16:12:28.028858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1416/2943] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:27 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:28.060832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:28.069776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:28.071314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:28.072474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a7718269-2ed5-4466-8a37-419e5afbc2ff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.168264 May 23 16:12:28.073023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1528/2944] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:28.827864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:28.828508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:28.834916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:28.835392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:28.836144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:28.836579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:28.919476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:28.920098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:28.927292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:28.927745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:28.928503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:28.928937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:28.961389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:28.970013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a01a86b0-f636-48ae-89a5-5e67a8a89c84 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145967 May 23 16:12:28.970576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1417/2945] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:28.992127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:29.000058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a11827d-3105-4d20-b579-5f98920709f6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084387 May 23 16:12:29.000621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1529/2946] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:28 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:29.044870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:29.045490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:29.051410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.051864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:29.052621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.053058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:29.091327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:29.091969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:29.099607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.100051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:29.100812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.101243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:29.226402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:29.234461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36bcfec9-c28f-4b65-b2a0-e82b496b403b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.193257 May 23 16:12:29.235284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1418/2947] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:29.257779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:29.266795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:29.268621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:29.270731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b212c619-9df9-4cac-8507-394d3b40a175 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.183024 May 23 16:12:29.271353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1530/2948] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:29.987493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:29.988143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:29.994619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.995073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:29.995867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:29.996303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.016806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:30.017325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:30.024007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.024438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.025183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.025617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.090869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.094909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:30.100068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17f1d197-4186-42e0-9432-1efb1a0a86a3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086773 May 23 16:12:30.100646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1531/2949] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:30.146374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.154258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24d66208-c77d-477f-b3c1-127320d481c1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.171151 May 23 16:12:30.154814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1419/2950] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:30.251556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:30.252169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:30.258418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.258864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.259642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.260075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.293708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:30.294244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:30.300940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.301394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.302158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:30.302596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:30.440008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.448413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-94e54c70-f1a5-4f68-b7ec-966f51676ade tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.200824 May 23 16:12:30.448984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1532/2951] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:30.465178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.474056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.475586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:30.476742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1dfb56e5-7a6d-4a4b-a40b-5353b27951bd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.186974 May 23 16:12:30.477316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1420/2952] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:31.117240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:31.117840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:31.124848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.125326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.126108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.126545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.172155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:31.172798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:31.175073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=96 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:31.179823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.184334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.184334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.184334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.187662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.195576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96e57947-9307-4049-9159-2b85153b3aa1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082114 May 23 16:12:31.196461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1533/2953] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:31.316855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.324737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-727e980e-357b-4f52-8257-9c5ec4af2fe6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.156931 May 23 16:12:31.325305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1421/2954] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:31.465753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:31.466384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:31.472348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.472789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.473535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.473970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.494608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:31.495269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:31.501769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.502226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.502999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:31.503469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:31.670845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.679432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a496372-85d5-4d43-a6bb-45a271d8c287 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217499 May 23 16:12:31.680029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1534/2955] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:31.691827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.700197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.701693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:31.702841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e72d6ce9-e568-4af2-81a4-78722b9f1bb9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212292 May 23 16:12:31.703398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1422/2956] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:32.212449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:32.213063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:32.220036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.220484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.221230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.221657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.292378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.299461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20f1b021-3608-4726-bf58-22f460837a5b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090839 May 23 16:12:32.300030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1535/2957] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:32.341547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:32.342158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:32.348680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.349125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.349871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.350309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.474553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.482458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-71c55148-b91b-4c26-bc0a-64e194a16e7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.144758 May 23 16:12:32.483016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1423/2958] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:32.696898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:32.697514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:32.703533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.704033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.704779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.705206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.721481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:32.722181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:32.728219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.728679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.729447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:32.729887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:32.884649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.892506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5052bd36-f4f1-4e9f-9b3f-a3ea852e2fa1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.199534 May 23 16:12:32.893517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1536/2959] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:32.905589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.914172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.915689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:32.916842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5c32169e-930f-424f-942b-8fdd17b25e53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.199280 May 23 16:12:32.917388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1424/2960] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:33.316777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:33.317393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:33.323843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.324288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.325035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.325468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.386254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:33.394531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c758a89-bc64-4f5b-ad45-88f85cd771e2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081570 May 23 16:12:33.395134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1537/2961] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:33.499961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:33.500516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:33.506977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.507470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.508234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.508672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.634157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:33.642642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9e3e8df-56dd-47f9-8c65-7b2d8666b64d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146547 May 23 16:12:33.643485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1425/2962] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:33 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:33.909956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:33.910502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:33.916513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.916962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.917750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.918187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.935623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:33.936237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:33.942616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.943070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:33.943878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:33.944295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:34.108590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.114959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.116848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-441c9606-c77c-44fd-b71e-4193accdb0d9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210791 May 23 16:12:34.117409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1538/2963] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:34.118401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:34.123769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.125860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.127023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f5977482-03a3-45e0-8a3d-bbad3a7a076d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.195317 May 23 16:12:34.127596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1426/2964] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:34.411419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:34.412030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:34.418432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:34.418875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:34.419704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:34.420121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:34.480631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.489008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-52512b9e-5c5c-4f76-8745-033892a2bd50 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081390 May 23 16:12:34.489565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1539/2965] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:34.666186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:34.666860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:34.674190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:34.674642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:34.675442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:34.675881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:34.803965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:34.811808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-878cd9f4-24b1-4966-b8df-6eb4b0b1626e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150337 May 23 16:12:34.812368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1427/2966] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:35.134218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:35.134762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:35.141261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.141701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.142450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.142882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.152006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:35.152629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:35.159314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.159767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.160537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.160974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.338398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.342214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.346503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6793beee-f4de-49dc-bc5c-7fe0219a5937 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216375 May 23 16:12:35.347078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1540/2967] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:35.351007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.352558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.353727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1851b53c-a74d-42fb-aeef-4704c2ffb0e3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.205641 May 23 16:12:35.354268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1428/2968] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:35.505003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:35.505640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:35.513354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.513792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.514550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.515680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.576911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.585399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6b7a524-7984-41dd-9a02-eed1c8a9e7ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084208 May 23 16:12:35.585959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1541/2969] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:35.829394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:35.829965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:35.836889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.837342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.838097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:35.838529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:35.962720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:35.970770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7dc7029d-fef1-4181-974a-e71c87923139 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145866 May 23 16:12:35.971359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1429/2970] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:35 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:36.364047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:36.364660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:36.370846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.371330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.371592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:36.372081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.372193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:36.372512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.378206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.378655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.379444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.379881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.560243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:36.564781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:36.568899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a53ce27c-dc69-4b47-9449-0747d8ec60b4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208668 May 23 16:12:36.569466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1542/2971] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:36.573177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:36.574673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:36.575862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-86a05cbd-1b50-48da-aa89-9ff856194fd1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.208065 May 23 16:12:36.576392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1430/2972] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:36.601325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:36.602060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:36.607924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.608366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.609622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.610050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.672626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:36.680402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee95d2e9-74a5-4cce-9be2-5a2bf23ea46f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082737 May 23 16:12:36.680957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1543/2973] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:36.988305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:36.988920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:36.995364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.995820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:36.996576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:36.997012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:37.120313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:37.128261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ca20ace-6e4b-4bd7-85d2-0f8cca90104e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.144361 May 23 16:12:37.128817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1431/2974] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:37.585929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:37.586548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:37.593136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:37.593524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:37.593657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:37.594140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:37.594326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:37.594761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:37.600252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:37.600710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:37.601468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:37.601905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:38.031053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.035006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.039149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e68724ff-2423-4903-8c0b-3240403808a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.457005 May 23 16:12:38.039752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1544/2975] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:38.043514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.044218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:38.044836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:38.045023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.046191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-185ec85a-9ec4-439d-8329-509ffa6bf773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.456520 May 23 16:12:38.046904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1432/2976] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:38.050953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:38.051440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:38.052294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:38.052729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:38.112267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.120071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-081e96fe-0027-41f8-a9f1-444616e1b677 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079389 May 23 16:12:38.120639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1545/2977] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:38.145885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:38.146505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:38.152697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:38.153151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:38.153917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:38.154358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:38.282744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:38.290596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e0a01889-c58f-444f-8412-d4569d7a7d29 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148780 May 23 16:12:38.291178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1433/2978] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:39.056367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:39.057011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:39.064206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.064605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:39.065026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.065245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:39.065396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.065834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.071693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.072137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.072887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.073318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.255110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.259168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.263361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77af9d13-78f7-40f5-a0f1-e423dd72ac04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210835 May 23 16:12:39.263939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1546/2979] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:39.268430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.269311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:39.269916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.270040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:39.271069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d4faf10-81ee-43dc-a2fe-fdf41708865b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.210883 May 23 16:12:39.271787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1434/2980] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:39.275630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.276087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.276882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.277318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.307689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:39.308331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:39.314825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.315300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.316059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:39.316490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:39.337514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.345386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee2e1219-15cb-4675-b768-50fe4141cf21 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080476 May 23 16:12:39.345948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1547/2981] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:39.442670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:39.450516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6f4ffb40-17e0-4565-8ce1-0d81e79166fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146743 May 23 16:12:39.451075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1435/2982] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:40.280511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:40.282394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:40.287901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:40.288522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:40.289237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.289697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.290496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.290936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.294988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.295474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.296241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.296716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.308770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:40.477200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.483999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.485474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c5704b22-6655-44eb-bd64-4dad9232f03a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208756 May 23 16:12:40.486138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1548/2983] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:40.490707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:40.491433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:40.492559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.494102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.495568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0263b218-698e-4c5a-b0c1-43a5278bd144 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.211507 May 23 16:12:40.496411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1436/2984] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:40.497754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.498214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.498985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.499456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.501095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:40.501706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:40.507501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.507957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.508757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:40.509196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:40.568116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.575973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04567352-1a3c-478b-a077-61720ce2943b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088824 May 23 16:12:40.576538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1549/2985] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:40.650480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:40.658484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c68e0c7-822b-4078-9dc4-2141d787d487 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160980 May 23 16:12:40.659053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1437/2986] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:40 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:41.503348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:41.503972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:41.510357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.510806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.511595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.512031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.513769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:41.514380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:41.521030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.521478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.522231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.522662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.711602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.718164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.720348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-924f1e7f-2652-4c1d-99c6-2200e9befb41 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220908 May 23 16:12:41.720926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1550/2987] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:41 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:41.725738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:41.726383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:41.727488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.729008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.730276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d9d7ecb2-b6a4-4f26-82b3-98f063881a54 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220542 May 23 16:12:41.730909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1438/2988] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:41.732401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.732852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.733666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.734107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.735306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:41.735922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:41.741768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.742220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.742985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:41.743492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:41.804090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.813043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee4b7990-9c89-4f4a-b806-43036d2065a2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091136 May 23 16:12:41.813617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1551/2989] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:41.883439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:41.891824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-072d6f31-4f3d-4600-b54d-ccad0953a645 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159949 May 23 16:12:41.892382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1439/2990] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:42.736624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:42.737240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:42.743779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.744228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.746127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.746588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.748450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:42.749083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:42.755875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.756319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.757208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.757640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.945406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:42.949278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:42.953749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ba403e2-c95c-4efa-835d-9fe99f92c8fa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.220953 May 23 16:12:42.954858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1552/2991] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:42.959494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:42.960879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:42.961024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:42.961737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:42.963033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-da2312c3-1971-408c-b15e-9f82e1248c6e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.218455 May 23 16:12:42.963727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1440/2992] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:42.966281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.966748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.967616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.968109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.981361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:42.982084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:42.988620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.989003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:42.989785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:42.990230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:43.038767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:43.048752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5cd1cccd-f216-4fd2-bfcd-62fdf3a65e2c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092747 May 23 16:12:43.049349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1553/2993] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:43.135350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:43.144194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5236de99-3b51-4499-a06f-0a1acaf9cd58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.171602 May 23 16:12:43.144768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1441/2994] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:43.972395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:43.972998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:43.979550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:43.980005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:43.980761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:43.981197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:43.981334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:43.981856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:43.988368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:43.988818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:43.989582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:43.990016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:44.176969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.179982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.184986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e8fad1c-6cb3-4b0b-a19e-e84c858fb88b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.217099 May 23 16:12:44.185564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1554/2995] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:44.188396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.189903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.190170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:44.191068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-64404889-ebcb-4ddc-81bd-3fbfe44e222a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.213727 May 23 16:12:44.191236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:44.191814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1442/2996] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:44.197037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:44.197491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:44.198255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:44.198694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:44.199626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:44.200240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:44.206727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:44.207202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:44.207969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:44.208409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:44.226704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:44.269239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.277289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae80fd4c-8ad7-458e-a44e-d296cdc0ead7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090775 May 23 16:12:44.277870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1555/2997] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:44.349519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:44.357364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6b49c536-5b1b-46ce-8712-1c2a6038981e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161443 May 23 16:12:44.357916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1443/2998] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:45.202080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:45.202679 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:45.209138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:45.209733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.209907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:45.210180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.210939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.211405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.216428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.216874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.217669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.218108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.403920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.407709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.411981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20d566f5-b850-4c45-b8b7-28e3d19da5a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.213857 May 23 16:12:45.412558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1556/2999] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:45.417280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.417429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:45.417972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:45.418213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.419436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ca428ed5-9cd2-4b32-962a-119888440437 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.214747 May 23 16:12:45.421658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1444/3000] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:45.423897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.424350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.424591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:45.425629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.425820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:45.426069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.431727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.432179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.432944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:45.433377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:45.497900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.506526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e6eb6e3-0975-439c-9a1a-ee86dc086497 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093018 May 23 16:12:45.507127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1557/3001] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:45.579760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:45.587990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8894f24-62e0-48ea-b6e3-806137b2ddb5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.167020 May 23 16:12:45.588593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1445/3002] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:45 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:46.429431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:46.429986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:46.436500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.436944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.437577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:46.438020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.438148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.438327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:46.445436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.445884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.446633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.447063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.625724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.633445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.634321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-495c480b-ef8d-4a51-aa8b-b717ca461286 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.209223 May 23 16:12:46.634920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1558/3003] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:46 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:46.641565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:46.642187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:46.648615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.650513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.651035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.651529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.651800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3ef7d238-486c-414f-9f7d-49a993b0890e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.218083 May 23 16:12:46.652308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.652427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1446/3004] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:46.652753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.657914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:46.657914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:46.664068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.664521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.665287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:46.665723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:46.723340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.731254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e74c389c-7a65-47f1-ba8c-902a8565a7d5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.095299 May 23 16:12:46.731833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1559/3005] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:46.807995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:46.815900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-275cdde7-d9b7-43bf-983f-67ac7e24c0d0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162613 May 23 16:12:46.816455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1447/3006] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:47.656356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:47.656897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:47.665667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.666116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.666888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.667352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.670166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:47.670801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:47.677696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.678125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.678883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.679347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.863722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:47.866974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:47.871698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67517878-0a75-4dbf-b466-6a309d47b87f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219269 May 23 16:12:47.872259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1560/3007] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:47.875696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:47.877297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:47.877542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:47.878827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:47.879327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0bc60db-650c-4214-acde-27bd645bbe19 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.212344 May 23 16:12:47.882686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1448/3008] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:47.884777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.885220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.885982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.886413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.887512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:47.888182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:47.894382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.894842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.895651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:47.896088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:47.955388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:47.964485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c031159a-a78e-457c-bc0b-2f7897c3f0ac tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090566 May 23 16:12:47.965068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1561/3009] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:48.039637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:48.047704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15594b9e-8f00-4867-8616-69cd5ba10599 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163931 May 23 16:12:48.048284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1449/3010] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:48.889157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:48.889774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:48.897116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:48.897575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:48.898306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:48.898735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:48.898735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:48.898957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:48.905331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:48.905885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:48.906650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:48.907090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:49.093695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.100587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.101570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f3a6b3a5-2dc6-44e5-930d-6fff5c4f3249 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.216226 May 23 16:12:49.102146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1562/3011] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:48 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:49.106537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:49.107170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:49.110910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.112991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.113351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:49.113803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:49.114550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89b67099-a187-47cb-8fcf-0211a6375a16 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.220916 May 23 16:12:49.114676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:49.114984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:49.115335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1450/3012] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:49.120099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:49.120360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:49.126392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:49.126847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:49.127652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:49.128094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:49.178138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.186305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-626acb77-225d-4f83-a060-2cdc59f42322 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083186 May 23 16:12:49.186889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1563/3013] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:49.277088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:49.285047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4546a269-4d64-4cd7-b6ad-f8be7f009b30 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.168732 May 23 16:12:49.285613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1451/3014] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:50.118419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:50.160448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:50.160448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.160448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.160448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.160913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.161010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:50.161010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:50.161010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.161010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.161733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.161733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.325645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.334278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eabe0957-9454-47d6-add4-290f426f9c95 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219611 May 23 16:12:50.334851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1564/3015] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:50.335690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.339337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:50.339946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:50.344576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.345547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.345993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.346123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.346743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.347227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.347350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a08d7f57-6a58-4d70-b6ce-bafc78bffc72 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.219031 May 23 16:12:50.347884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1452/3016] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:50.352486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:50.352650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:50.353283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:50.359054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.359543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.360407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:50.360943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:50.419303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.428367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fccd8fb1-565b-4a59-bcfc-1ab64ac13e89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092496 May 23 16:12:50.429134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1565/3017] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:50.500212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:50.508704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c7f8026-631f-4207-a46b-91d75cf8fcdb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159899 May 23 16:12:50.509265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1453/3018] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:51.351509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:51.352182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:51.359011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.359489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.360242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.360756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.366117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:51.366758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:51.373122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.373569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.374322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.374754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.556958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.563138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.566507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-efd04e36-2bc6-4324-912a-a3bae79e970c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.218312 May 23 16:12:51.567206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1566/3019] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:51.571766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:51.572862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:51.576698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.578249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.578409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.578691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.579481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.579598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5d6cf93-dc59-4f51-aa1b-2a85098c98ce tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.217710 May 23 16:12:51.584258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.584408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1454/3020] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:51.584613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:51.585413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:51.591272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.591938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.592711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:51.593153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:51.642831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.651158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2b51eeba-a92b-463d-93d3-2bd7f89df414 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082943 May 23 16:12:51.651737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1567/3021] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:51.739735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:51.747798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-30eb4d5f-da03-4fbe-94d5-7ceea6234d4e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.166708 May 23 16:12:51.748305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1455/3022] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:51 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:52.584263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:52.584905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:52.591360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.591805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.592557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.592986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.601073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:52.601687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:52.607767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.608220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.608974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.609407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.773273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.781492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-062a36a8-b349-4504-8469-2db3dc3f0f19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.201681 May 23 16:12:52.782062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1568/3023] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:52.786398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:52.787428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:52.792825 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.793266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.794019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.794478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.794618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.803152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.804660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.805885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-67a4fbc6-bdae-4fcc-8320-bca5de541921 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.208685 May 23 16:12:52.806445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1456/3024] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:52.810813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:52.811453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:52.817595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.818045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.818806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:52.819274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:52.857573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.866804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ab20ecf9-b0b8-4d90-a0c0-bbca0b798070 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083773 May 23 16:12:52.867402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1569/3025] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:52.950325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:52.958355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a293bb9e-7776-4640-a796-6b0bc1177d50 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150977 May 23 16:12:52.958910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1457/3026] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:53.797699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:53.798342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:53.804814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:53.805254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:53.805998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:53.806423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:53.824757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:53.825367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:53.832047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:53.832501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:53.833261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:53.833693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:54.008288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.014890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.016223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3cba0187-9a5f-4975-952d-bea315c280fb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.222355 May 23 16:12:54.016801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1570/3027] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:54.022200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:54.022807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:54.024865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.026482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.027692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-514accc7-55ff-45a4-81b9-d9e71446144b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.206778 May 23 16:12:54.028247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1458/3028] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:54.028573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:54.029026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:54.029788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:54.030225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:54.032905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:54.033416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:54.039325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:54.039803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:54.040548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:54.040987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:54.099556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.107545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ef8d371d-2084-4bd6-83e6-11d20787d366 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089549 May 23 16:12:54.108114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1571/3029] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:54.183415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:54.191816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-94dab43e-35f0-48b7-8eb1-32fdb26eb463 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162435 May 23 16:12:54.192380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1459/3030] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:54 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:55.033918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:55.035140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:55.041530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.041971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.042722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.043179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.045647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:55.046268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:55.053083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.053549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.054339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.054782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.059410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:55.196436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.205522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.207118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.208319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae3a59c1-f799-42c0-b5b6-890dd565defa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.166615 May 23 16:12:55.208878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1460/3031] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:55.213291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:55.213903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:55.220341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.220789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.221559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.221998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.306601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.314489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fd762be4-079f-4a8b-8fdd-31bb20d59f2a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.104631 May 23 16:12:55.315055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1461/3032] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:55.320157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:55.320764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:55.326650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.327124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.327895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:55.328332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:55.367076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.375175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d54ddcbf-ef08-4478-9864-01b7f5d62e04 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.345340 May 23 16:12:55.375742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1572/3033] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:55.460048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:55.467949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7aab82d9-ec80-4a6b-a105-1384e4dbdb8e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151248 May 23 16:12:55.468512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1462/3034] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:56.227663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:56.228292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:56.235125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.235602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.236368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.236785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.331673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:56.332909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:56.339400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.339852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.340610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.341041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.367613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.376048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.377539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.378691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56ee31cc-0d3d-4d79-aeba-d81610f261b8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.155285 May 23 16:12:56.380039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1573/3035] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:56.394620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:56.395817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:56.402043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.402562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.403424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.403956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.404087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.411949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17d2743b-8f26-48e4-8548-0a3803dec20f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084115 May 23 16:12:56.412514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1463/3036] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:56.485427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:56.486051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:56.492149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.492601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.493365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:56.493805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:56.531051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.539005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-309223be-f4b3-47bc-b8bf-2caf7ccafaaa tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148221 May 23 16:12:56.539602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1574/3037] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:56.625135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:56.633729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1414b8ef-789f-42e6-a600-eb5a7caea1b4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152104 May 23 16:12:56.634296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1464/3038] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:56 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:57.398174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:57.398786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:57.405269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.405712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.406458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.406888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.428587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:57.429310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:57.435889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.436345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.437100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.437600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.516457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.525134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39e3f0e0-0640-4238-904f-5b9ac84d71d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.100394 May 23 16:12:57.525703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1465/3039] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:57.553321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.556657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:57.557273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:57.561731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.563123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.563244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.563616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.564367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.564475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7170bdee-a846-4c1c-9c55-68e5721d4f3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170972 May 23 16:12:57.564803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.565562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1575/3040] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:57.651743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:57.652362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:57.658324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.658772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.659556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:57.659991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:57.694826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.702835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-58aafa3c-cd6b-4126-aaf3-7d9bfbca3954 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149951 May 23 16:12:57.703430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1466/3041] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:57.790689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:57.795670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-97a94598-22cb-47bd-b8fd-967fc674f875 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147756 May 23 16:12:57.796239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1576/3042] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:58.541709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:58.542324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:58.548766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.549212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.549957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.550388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.581793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:58.583217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:58.585755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=98 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:58.590642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.591283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.592135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.592651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.610710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.618878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c42025fe-da76-4637-94cb-369f9a7b4d89 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080973 May 23 16:12:58.619465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1467/3043] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:58.719746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:58.720480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:58.726431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.726868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.727665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.728106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.729264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.737631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.739173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.740356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a17b939a-28ad-496b-8eb8-fcef065bb481 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.162503 May 23 16:12:58.740916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1577/3044] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:12:58.812634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:58.813259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:58.819875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.820332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.821097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:58.821534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:58.852813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.860893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae77675b-03e3-42ff-b578-72cf026d7d7f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144895 May 23 16:12:58.861557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1468/3045] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:58.950436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:58.958293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3eb8243e-c368-484b-a3a3-92bbfa3b5a0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149626 May 23 16:12:58.958850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1578/3046] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:12:59.635797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:59.636359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:59.642862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.643601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.644369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.645313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.704672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:59.713235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6dc7a219-f59e-45ce-80bd-ec5fd0c926ae tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081378 May 23 16:12:59.713819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1469/3047] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:12:59.758301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:59.758913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:59.765297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.765745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.766500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.766935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.878107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:59.878734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:59.884744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.885199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.886026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.886414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.905223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:59.922155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:59.923765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:12:59.924962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9bdc6983-bd8a-4d8b-8f8d-ad293778508e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170548 May 23 16:12:59.925523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1579/3048] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:12:59 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:12:59.975047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:12:59.975691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:12:59.981664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.982109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:12:59.982870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:12:59.983335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:00.021497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:00.029551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1aa220e3-0acc-428a-a808-a51eeed5c8bf tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.155206 May 23 16:13:00.030125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1470/3049] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:00.114786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:00.122727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b4f08b3a-596b-4a5f-8945-71bec1825111 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151468 May 23 16:13:00.123319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1580/3050] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:12:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:00.729982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:00.730599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:00.732946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:00.737595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:00.738061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:00.739508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:00.739940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:00.802053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:00.810066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f35d5505-b0e3-4263-a853-e47954a1ac0e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084028 May 23 16:13:00.810625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1471/3051] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:00.944371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:00.944983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:00.951503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:00.951957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:00.952715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:00.953151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.047407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:01.048051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:01.054519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.054971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.055755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.056189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.081210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.085196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:01.090587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.092116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.099548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-02947f30-66ef-488f-b7ea-be9687b9594d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.152902 May 23 16:13:01.099548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1581/3052] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:01.140021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:01.140638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:01.147945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.148396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.149179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.149612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.180394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.188468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d3c0e13-b55d-4cdc-a9a0-0965511183cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144868 May 23 16:13:01.189038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1472/3053] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:01.277440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.285409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-62835ef2-059e-4326-b4f8-20de83c95ab1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149060 May 23 16:13:01.285912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1582/3054] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:01.826834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:01.827491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:01.834160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.834610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.835421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:01.835856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:01.896972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:01.904907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f82756e6-490a-42bf-b40b-19226cf2a9d7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081960 May 23 16:13:01.905468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1473/3055] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:02.112598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:02.113139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:02.120054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.120498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.121242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.121671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.204333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:02.204957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:02.211073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.211569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.212357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.212801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.248987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.257433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.258953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.260163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c37ff8fb-1f1a-41db-a089-227117389e6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.151613 May 23 16:13:02.260752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1583/3056] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:02.304062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:02.304685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:02.310895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.311385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.312182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.312651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.347561 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.356377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-48448778-58de-455e-8e15-249a1b7e2065 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.155853 May 23 16:13:02.356970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1474/3057] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:02 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:02.445402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.453233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b3110c9-4470-42fd-b5c5-5b0621bb9bbd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154372 May 23 16:13:02.453793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1584/3058] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:02 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:02.921644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:02.922258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:02.928740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.929183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.929929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:02.930358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:02.991040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:02.999000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1fa6e99a-830e-4f16-948d-8ad60bfdf8e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081296 May 23 16:13:02.999588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1475/3059] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:03.278695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:03.279341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:03.285931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.286378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.287162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.287600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.374675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:03.375315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:03.382633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.383088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.383884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.384324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.432943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:03.442468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:03.444057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:03.445279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47109f83-5c57-40df-9f0b-356b88de1b66 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.170543 May 23 16:13:03.446157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1585/3060] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:03.469945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:03.470567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:03.476719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.477171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.477934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:03.478370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:03.529151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:03.532600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:03.537937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8640f6a0-e9ae-4806-a6fc-3fa93be8bb92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.167580 May 23 16:13:03.538550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1476/3061] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:03.611337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:03.620278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c7508ea4-5abc-459f-846e-14561de367f2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154423 May 23 16:13:03.620841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1586/3062] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:04.017705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:04.019125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:04.026555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.027006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.027875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.028308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.089665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.097621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-611e3af8-88ae-4b7b-bebd-ed578659c538 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085145 May 23 16:13:04.098792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1477/3063] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:04.464420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:04.464980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:04.471521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.471976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.472773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.473215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.554912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:04.555551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:04.561529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.561974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.562725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.563184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.600549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.609282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.610818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.612023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6eb6a17d-1486-405d-a37c-b30b532099f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1799 microversion: 2.20 time: 0.151714 May 23 16:13:04.617904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1587/3064] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1799 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:04.637674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:04.638293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:04.645158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.645647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.646429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:04.646869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:04.726182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.734202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02735e72-1be8-4983-b35e-0879a914a2ee tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.183051 May 23 16:13:04.734888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1478/3065] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:04.786231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:04.794200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-195771c3-09f6-4bad-a0c7-86fd2d73c074 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160329 May 23 16:13:04.794762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1588/3066] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:05.115886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:05.116613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:05.123479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.123942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.124714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.125160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.189592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:05.198409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb88a0b5-e25e-4696-ab30-60e6bb7bf220 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086951 May 23 16:13:05.199858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1479/3067] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:05 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:05.629979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:05.630608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:05.636996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.637444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.638202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.638637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.751643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:05.752600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:05.760771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.761639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.762539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:05.762977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:05.963404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:05.981624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:05.989365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:05.989365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3e2693fb-ec30-442d-bfe7-147b4616de69 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1732 microversion: 2.20 time: 0.358662 May 23 16:13:05.991958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1589/3068] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1732 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:05.997494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:05.998106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:06.005150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:06.005609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:06.006363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:06.006927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:06.058518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:06.066338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70477f55-a3b4-4113-85f3-6acb94445b30 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.318476 May 23 16:13:06.077807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1480/3069] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:06.087970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-fccb767d-082a-4559-94f7-b65efca2de52 req-14c69491-4df5-4302-94f8-e4621f8a371f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6af85f59-fe73-4757-ab1a-d0ad26e6bd0d", "name": "network-vif-deleted", "tag": "51745985-c515-4c56-8856-7b32a9d291ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:13:06.094894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-fccb767d-082a-4559-94f7-b65efca2de52 req-14c69491-4df5-4302-94f8-e4621f8a371f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:06.095375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-fccb767d-082a-4559-94f7-b65efca2de52 req-14c69491-4df5-4302-94f8-e4621f8a371f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:06.108690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-fccb767d-082a-4559-94f7-b65efca2de52 req-14c69491-4df5-4302-94f8-e4621f8a371f service nova] Creating event network-vif-deleted:51745985-c515-4c56-8856-7b32a9d291ed for instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d on hv-cinder-83588 May 23 16:13:06.116111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-fccb767d-082a-4559-94f7-b65efca2de52 req-14c69491-4df5-4302-94f8-e4621f8a371f service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036052 May 23 16:13:06.117268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1481/3070] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:13:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:13:06.203042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:06.209948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-033de7ee-4666-40ad-8fb4-53660b3cd50c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.215548 May 23 16:13:06.209948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1590/3071] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:05 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:06.222807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:06.223579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:06.232585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:06.233254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:06.237202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:06.238069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:06.303534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:06.311762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd172932-3248-4560-8c81-70c76c36bf53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.099011 May 23 16:13:06.312585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1482/3072] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:07.003001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:07.003647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:07.010045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.010494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.011276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.011713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.072428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.080893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.083040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.088853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-42f2ecc5-993a-414f-92ea-b15528fcd303 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085226 May 23 16:13:07.089000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:07.089121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1591/3073] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:06 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:07.089224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:07.092463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.092919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.093691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.094130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.224861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.226482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:07.227219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:07.237402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.237402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.237402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.237402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.238401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a19e5586-55e3-4b35-8939-e1b35e209fc6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.157620 May 23 16:13:07.238873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1483/3074] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:07.328786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:07.329399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:07.335991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.336378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.337124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:07.337556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:07.366892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.374853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8763db04-258a-4fea-abbc-c5bfd45e1822 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152984 May 23 16:13:07.375453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1592/3075] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:07.397104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:07.404922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-86b9911e-8aa7-4768-82ee-85c2f89f83f9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079963 May 23 16:13:07.405478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1484/3076] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:08.102423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:08.103166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:08.109654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.110098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.110846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.111319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.171544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.180302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.181792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.182925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8309c60e-bb95-4757-8fd8-03de00899a90 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084397 May 23 16:13:08.183512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1593/3077] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:08.254922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:08.255594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:08.262087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.262540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.263330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.263773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.389688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.391776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:08.392389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:08.397898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b5c7596e-7bd4-44a3-a310-c1bed4e6b079 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146872 May 23 16:13:08.398459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1485/3078] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:08 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:08.398580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.398898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.399682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.400114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.421787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:08.422400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:08.429320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.429764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.430523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:08.430953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:08.497941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.506338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-407bcef7-ff0b-44c5-ae2c-4bf25e48a5a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088397 May 23 16:13:08.507319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1486/3079] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:08.555914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:08.563734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-340d02b2-18b7-4372-a53d-dfc4053d113b tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.175787 May 23 16:13:08.564293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1594/3080] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:09.200885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:09.201505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:09.208729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.209110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.209871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.210346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.270457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.278862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.280391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.281533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-96b8d8fa-a34b-445a-b8ff-a78736843570 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084596 May 23 16:13:09.282088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1487/3081] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:09.414459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:09.415069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:09.421589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.422034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.422785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.423266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.523685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:09.524301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:09.530106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.530551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.531334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.531764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.548153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.556810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24b5d8d3-51ce-4662-84a7-90ef4c0109a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146172 May 23 16:13:09.557567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1595/3082] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:09.581170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:09.581790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:09.587789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.588239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.588998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:09.589441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:09.596644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.604691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f296eab4-9ea6-4732-84ff-b404f2426f6d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084769 May 23 16:13:09.605255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1488/3083] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:09.716185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:09.723325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2549b086-2aef-48d4-a719-e38c6a94bc37 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145650 May 23 16:13:09.723976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1596/3084] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:10.299492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:10.300105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:10.306598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.307043 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.307821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.308252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.368986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.377188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.378678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.379839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-79778d43-468f-4d76-ae1f-79449fb9d2a6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084230 May 23 16:13:10.380395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1489/3085] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:10.573796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:10.574335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:10.580834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.581293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.582680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.583140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.621584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:10.622290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:10.628355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.629379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.630067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.630500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.701793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.710252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b2d15632-4671-45d9-83ab-c3c5f84082ba tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092463 May 23 16:13:10.710810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1490/3086] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:10.730777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.739706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c30dc837-af09-40b3-ae8b-30bf5a415d67 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.169649 May 23 16:13:10.740492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1597/3087] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:10.743130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:10.743754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:10.749942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.758533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.758533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:10.758533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:10.877283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:10.885839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-42039250-ad1b-4334-9714-e9aabd5cc906 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147374 May 23 16:13:10.886401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1491/3088] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:10 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:11.398162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:11.398940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:11.404998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.405439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.406184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.406614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.424464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:11.466256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:11.474921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:11.476436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:11.477566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d47440c4-df17-47f6-af88-7485a1c2f1c6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083683 May 23 16:13:11.478124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1598/3089] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:11.727580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:11.728110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:11.734648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.735116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.735900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.736341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.756716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:11.757444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:11.763810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.764261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.765019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.765459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.802528 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:11.810720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-96bef754-de6b-4b3e-91cd-58589c394388 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087551 May 23 16:13:11.811290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1492/3090] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:11.899551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:11.903343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:11.903974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:11.907515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-96f824bc-dd26-47bc-ab19-48711eee363f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.154698 May 23 16:13:11.908120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1599/3091] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:11.910056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.911171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:11.911970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:11.912409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.048981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:12.056990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9bf33ee9-6bf3-4054-9802-e2b6173f4e71 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157588 May 23 16:13:12.057559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1493/3092] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:12.495471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:12.496092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:12.502131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.502582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.503482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.503937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.563748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:12.572808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:12.574325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:12.575514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a7c3118-416f-47e4-b207-5a64e587b456 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083967 May 23 16:13:12.576076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1600/3093] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:12.828096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:12.828632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:12.835102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.835577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.836335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.836773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.840747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:12.897518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:12.905313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a00fcdfd-74ae-46b2-ba02-e481dea3905f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081276 May 23 16:13:12.905856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1494/3094] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:12.925125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:12.925776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:12.932751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.933209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:12.933975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:12.934414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.060010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:13.067875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad5e2e1d-28fd-448e-8bc1-863b84594f6e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146646 May 23 16:13:13.077523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1601/3095] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:13.077716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:13.077716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:13.080995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.081398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.082671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.083139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.212386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:13.220340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d4bc855a-0201-4fcf-9e49-bf8fb56566fa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150368 May 23 16:13:13.220907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1495/3096] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:13.593608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:13.594225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:13.600305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.600757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.601747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.601912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.662267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:13.670814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:13.672355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:13.673499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5068a800-6cd2-4e90-b7a4-65062a6ed7cb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083852 May 23 16:13:13.674146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1602/3097] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:13.923985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:13.924603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:13.931650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.932102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.932873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:13.933306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:13.993755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.001593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-759580e8-3cd9-459a-be40-a5ede53ac487 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081459 May 23 16:13:14.002150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1496/3098] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:14.084059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:14.084669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:14.090972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.091447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.092193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.092620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.221014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.229304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0038f0a8-2e2c-4a2d-b38c-98214738e171 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149085 May 23 16:13:14.229866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1603/3099] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:14 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:14.244029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:14.244652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:14.250734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.251228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.251992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.252429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.378286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.386195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a8faf66-3580-424b-b9fc-dccaf868fe7f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152254 May 23 16:13:14.386760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1497/3100] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:14.691871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:14.692483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:14.698518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.698967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.699748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:14.700186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:14.761125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.770295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.771840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:14.772975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad42060d-e4be-4d50-8f6a-10a2b99bb91a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084988 May 23 16:13:14.773536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1604/3101] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:15.018152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:15.019017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:15.026044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.026488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.027268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.027701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.091351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.099259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-85dc4c0a-57bd-4318-9276-c4ea0261211e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084932 May 23 16:13:15.099824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1498/3102] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:15.246953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:15.247607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:15.254529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.254981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.255787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.256210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.381776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.389768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-82bc495e-a361-4bef-99ed-547b4c91217f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146733 May 23 16:13:15.390339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1605/3103] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:15.403166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:15.403786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:15.409807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.410258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.411021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.411486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.539765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.547569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-875ee843-c779-4feb-aa1a-4d07b068a601 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148188 May 23 16:13:15.548205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1499/3104] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:15.791036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:15.791592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:15.797537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.797984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.798730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:15.799205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:15.859595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.868453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.869936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:15.871063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a740d4f2-befc-4fc0-8441-cb6838cd22f1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083994 May 23 16:13:15.871636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1606/3105] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:16.115159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:16.115730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:16.122370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.122821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.123661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.124092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.184762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.193015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15fa84e7-85ff-48f2-8da9-45464eabf7a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081892 May 23 16:13:16.193643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1500/3106] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:16.407167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:16.407696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:16.414664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.415189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.415900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.416335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.539265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.547021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-861bab55-a027-4064-b340-b52fa501b55a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.143901 May 23 16:13:16.547620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1607/3107] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:16.564461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:16.565072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:16.571148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.571600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.572345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.572777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.699900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.707872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f3190909-27a9-493e-b70f-5f81774b9f7a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147117 May 23 16:13:16.708436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1501/3108] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:16 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:16.888872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:16.889479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:16.896154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.896597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.897341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:16.897770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:16.956803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.965438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.966930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:16.968087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b71f7085-f854-4729-8f23-efe245a07f08 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083081 May 23 16:13:16.968643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1608/3109] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:17.210012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:17.210552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:17.216683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.217138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.217890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.218325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.279918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:17.287762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d6528477-34bc-40ed-a01b-08e242db7130 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082337 May 23 16:13:17.288336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1502/3110] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:17 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:17.564196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:17.564818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:17.571973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.572426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.573185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.573622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.701128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:17.709442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-443d1545-876c-43b2-8957-5fd25d45547d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149075 May 23 16:13:17.710004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1609/3111] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:17 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:17.725714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:17.726335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:17.732354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.732837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.733599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.734329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.859008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:17.867270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73acb7f2-89cc-42cf-952a-b23222ef2bea tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146169 May 23 16:13:17.867836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1503/3112] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:17.986436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:17.987051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:17.993037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.993493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:17.994288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:17.994731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.054444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:18.063193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:18.064708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:18.065863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fd963941-aea9-48b6-b352-d40105ba0890 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083363 May 23 16:13:18.066938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1610/3113] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:17 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:18.305010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:18.305630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:18.312160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.312613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.313369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.313806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.376733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:18.385332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f0d1a67b-808d-4580-ade2-fe1d2500fc15 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084772 May 23 16:13:18.385894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1504/3114] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:18.726973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:18.727613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:18.734021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.734465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.735250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.735684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.862468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:18.871198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbcff6f4-fa05-4f9d-84a0-ad170e90912c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148589 May 23 16:13:18.871773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1611/3115] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:18 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:18.884055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:18.884675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:18.890520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.890966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:18.891768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:18.892206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.025804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:19.033937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2bf622a5-1dd5-42ab-a87d-5cc79a449d5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.153620 May 23 16:13:19.034502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1505/3116] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:19.089574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:19.090513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:19.100378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.100835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.101611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.102051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.175047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:19.183307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:19.184803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:19.185931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f301bbbd-d7cb-42f5-9145-e756c39888f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.102660 May 23 16:13:19.186485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1612/3117] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:19.402506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:19.403047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:19.409496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.409991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.411302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.411737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.475570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:19.483879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a916dba-593f-423e-8c4c-785a93243e22 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085240 May 23 16:13:19.484438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1506/3118] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:19.888828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:19.889450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:19.896542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.897059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:19.897817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:19.898250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.031517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.039760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bc1c9e6-c790-466e-8d49-e3b884296b3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.154805 May 23 16:13:20.040335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1613/3119] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:20.051878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:20.052500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:20.059499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.060927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.061712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.062156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.194661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.202759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e2de322-538e-4f65-93ec-7d55475c2468 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155245 May 23 16:13:20.202952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:20.203351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1507/3120] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:20 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:20.203469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:20.209396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.209839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.210684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.211136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.272022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.280308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.281795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.282930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b4d5f005-ba56-4f9b-af69-ea89b660cddd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083925 May 23 16:13:20.284826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1614/3121] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:20.500641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:20.501243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:20.507747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.508196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.508947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:20.509380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:20.569102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:20.577017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-be9fac18-48c4-4e39-9bdd-6b3005259d72 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080257 May 23 16:13:20.577573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1508/3122] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:21.056586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:21.057202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:21.063841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.064288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.065035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.065467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.069858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:21.194252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.201987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dfc380cf-3606-40cb-8610-7c16e27f880e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149238 May 23 16:13:21.202543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1615/3123] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:21 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:21.220207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:21.220847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:21.226789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.227271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.228019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.228451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.301432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:21.302094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:21.309284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.309284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.309720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.309813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.385563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.388724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.394437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.396049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.396934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8918be63-cddf-4540-ac84-77948da2be75 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.180500 May 23 16:13:21.397296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0978765d-9e36-482d-9269-5189794e4f6a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.100373 May 23 16:13:21.397500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1509/3124] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:21.402417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1616/3125] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:21.594952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:21.595617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:21.602296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.602748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.603555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:21.603997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:21.670321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:21.673821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:21.678673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b3261ee0-217c-4ece-b0a4-963e09e25397 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088747 May 23 16:13:21.679271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1510/3126] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:22.219354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:22.219971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:22.226565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.227017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.227814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.228261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.352640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.360512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a42d56d7-a05f-457f-be18-5cd6ae6c9e0a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.145079 May 23 16:13:22.361080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1617/3127] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:22.413750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:22.414097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:22.414518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:22.414714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:22.420647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.420982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.421115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.421431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.421838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.422180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.422305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.422614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.510776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.519439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.520932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.522067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dce90526-c759-4ab3-8801-af77810b979d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.111748 May 23 16:13:22.522621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1618/3128] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:22.572909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.580792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0f43805b-8313-4705-9798-e3ac1c74154f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.170932 May 23 16:13:22.581354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1511/3129] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:22.694744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:22.695382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:22.701373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.701826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.702580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:22.703017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:22.762451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:22.770507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f5ba3cf5-2ad3-4726-8f7e-20fb564445df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079635 May 23 16:13:22.771072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1619/3130] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:23.377792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:23.378412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:23.384911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.385371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.386122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.386564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.510755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.518617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77d179b7-980d-42a0-84bc-0ea4c76a77cb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144815 May 23 16:13:23.519469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1512/3131] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:23.540481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:23.541100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:23.547577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.548031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.548798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.549235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.598554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:23.599198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:23.605161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.605617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.606375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.606812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.613533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.622164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.623716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.624862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cbae30e4-8bdd-43ac-8db7-4b680b7b5a1a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088282 May 23 16:13:23.625419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1620/3132] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:23.737135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.744968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b2302cd8-2a60-442e-9eee-cce92f2fdbc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150831 May 23 16:13:23.745519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1513/3133] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:23.787796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:23.788340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:23.794173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.794618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.795396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:23.795830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:23.857082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:23.865099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1e188328-0da1-4586-b8ac-607458bf6717 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081884 May 23 16:13:23.865674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1621/3134] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:23.953742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:13:23.954016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:13:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:13:24.535865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:24.536500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:24.542956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.543442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.544207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.544647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.632910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:13:24.633077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:13:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c (ip 10.222.0.32) !!! May 23 16:13:24.642741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:24.643381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:24.649835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.650281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.651028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.651483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.676624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.684880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36201d0b-a961-4b3a-ad2c-a981ad46b018 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152883 May 23 16:13:24.685452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1514/3135] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:24.711996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.720392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.721910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.723594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-65343675-c5f8-497d-840b-374115dd5f98 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084683 May 23 16:13:24.724151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1622/3136] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:24.761694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:24.762315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:24.768286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.768988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.769738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.770173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.881972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:24.882613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:24.888612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.900199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.900199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:24.900199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:24.900629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.903342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5de20e7-1a07-471d-8490-55bf5b4a13fd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145398 May 23 16:13:24.903910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1515/3137] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:24 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:24.955880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:24.964691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0b269ab5-65a2-4cae-a402-78c2c6debc5d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086852 May 23 16:13:24.965268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1623/3138] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:25.702335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:25.702897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:25.709713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.710164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.710921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.711397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.744048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:25.744753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:25.751890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.752423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.753197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.753635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.829909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:25.838309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:25.839862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:25.841011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc82619e-d38e-4d0e-a0cc-be70be4c98db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.100811 May 23 16:13:25.841572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1624/3139] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:25.858864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:25.867324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-632e53c1-f18b-4d9c-8f4f-d6a35fcae7a3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.168980 May 23 16:13:25.867884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1516/3140] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:25.920871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:25.921485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:25.927946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.928397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.929146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.929580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.981549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:25.982303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:25.988406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.988865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:25.990148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:25.990569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:26.064481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:26.073092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-968f4b11-c62d-462d-a045-3e05e3abcec5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.095406 May 23 16:13:26.073670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1517/3141] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:26.093299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:26.101197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1c5d5124-ffe9-44bf-b43f-684eeb82ae5d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.184180 May 23 16:13:26.101752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1625/3142] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:25 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:26.859146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:26.859767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:26.866389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:26.866841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:26.867624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:26.868063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:26.884745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:26.885359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:26.891928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:26.892379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:26.893181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:26.893619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:26.930696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:26.939218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:26.940724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:26.941875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c6a037f6-5525-4e9b-bf47-fe8ac83ea1fc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086696 May 23 16:13:26.942439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1518/3143] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:27.021841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:27.029681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2c6b3392-5949-455e-bab9-74ca7e81467e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149241 May 23 16:13:27.030241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1626/3144] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:27.090129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:27.090776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:27.096764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.097217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:27.097983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.098420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:27.117974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:27.118517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:27.120324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:27.124987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.125432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:27.126180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.126615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:27.164340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:27.172512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-31d4a18e-13d4-4ebe-be0b-8540cc3423c1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086011 May 23 16:13:27.173094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1519/3145] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:27.257132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:27.265085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36edd14f-649a-428e-ae38-f445a5dfe71d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150992 May 23 16:13:27.265642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1627/3146] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:27.960798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:27.961840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:27.968079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.968536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:27.969296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:27.969735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.032158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.041316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.042846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.044020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e85af14c-3eaa-4dae-913b-30d1073a6fe0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087242 May 23 16:13:28.044578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1520/3147] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:27 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:28.045847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:28.046472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:28.052931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.053378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.054139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.054575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.181653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.189301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:28.189924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:28.190131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b16eb6e2-98f6-47ea-b438-166fe4f0cb05 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148271 May 23 16:13:28.190707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1628/3148] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:28 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:28.192848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:28.196409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.196859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.197618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.198060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.258962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.266941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-27a9dd1c-26c4-4031-bd18-8fbb539ead1f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081468 May 23 16:13:28.267525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1521/3149] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:28 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:28.282737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:28.283379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:28.289943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.290392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.291173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:28.291611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:28.414920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:28.422802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b2f024af-e2c6-4e2b-bbb6-e137090fb8d6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.143901 May 23 16:13:28.423396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1629/3150] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:29.061802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:29.062422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:29.069780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.070224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.070972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.071437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.135792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.144780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.146328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.147853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e42d3f02-5314-41ba-a6df-bedab10e886d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089988 May 23 16:13:29.148485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1522/3151] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:29.206934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:29.207571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:29.213993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.214435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.215208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.215637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.283980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:29.284598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:29.291191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.291507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.292266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.292706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.361606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.365568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.370221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-034c544d-82a6-4eaf-98cf-6ba9a7a6ff8a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090100 May 23 16:13:29.370873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1523/3152] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:29.373445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da600e5d-b95a-4ab0-9054-24bdfa25c400 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.170267 May 23 16:13:29.373995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1630/3153] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:29.439439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:29.440064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:29.446045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.446498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.447294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:29.447738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:29.575575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:29.584018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7094d122-f1b4-4a8b-9f8d-c743fb912891 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147698 May 23 16:13:29.584585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1524/3154] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:30.166075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:30.166724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:30.172937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.173388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.174149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.174585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.234258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.237805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:30.243576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.246854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.246854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-98c71b02-ba39-4385-8234-2a01c2370cfc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084143 May 23 16:13:30.246854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1631/3155] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:30.387238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:30.387862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:30.389737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:30.390385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:30.394677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.395158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.395964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.396404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.396740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.397190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.397943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.398376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.480939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.488867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0d4f6190-665a-4703-bc69-f647ec39ec6e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.105558 May 23 16:13:30.489947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1525/3156] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:30.547220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.555576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78fbd2aa-e1cd-4d9f-8262-76f4cb4b97cc tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.169933 May 23 16:13:30.556139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1632/3157] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:30.600879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:30.601497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:30.607507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.607967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.608728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:30.609168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:30.738641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:30.744426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a987e3f-1c0f-40ab-b5f1-292af4f95cd4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147348 May 23 16:13:30.744995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1526/3158] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:30 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:31.265033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:31.265646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:31.272933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.273376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.274124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.274557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.334554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.342805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.344353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.345489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e6a813c-5fe1-40e7-86a7-5b08120a09ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084481 May 23 16:13:31.346039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1633/3159] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:31.506478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:31.507041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:31.513835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.514280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.515027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.515554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.533957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:31.573106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:31.573721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:31.579787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.580231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.580626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.580985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.581412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.588725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83eefbb3-a3e2-453d-80f7-112c407aaced tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086130 May 23 16:13:31.589397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1527/3160] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:31.707788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.715750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67ebe36a-02cd-44b6-ab0a-04b136a1f2b3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146817 May 23 16:13:31.716308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1634/3161] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:31.761271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:31.761976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:31.767966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.768411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.769418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:31.769848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:31.892943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:31.901634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-02a0e67d-8e43-449d-ab6b-ff975bc846b7 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.144239 May 23 16:13:31.902189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1528/3162] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:32.364008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:32.364622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:32.371110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.371569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.372315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.372741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.435530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:32.444350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:32.445841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:32.446975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c653a35-f10b-42dd-ba2c-cadeee67ad60 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086864 May 23 16:13:32.447553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1635/3163] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:32.605984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:32.606532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:32.613040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.613493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.615080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.615549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.676032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:32.684613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d4a3b71d-62e1-45c9-a97c-fc5529b79eea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082666 May 23 16:13:32.685174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1529/3164] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:32.732708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:32.733426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:32.739306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.739752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.740503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.740935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.879125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:32.887726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-17735458-3293-440c-8fb0-f472d737a241 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.158528 May 23 16:13:32.888089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1636/3165] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:32.917733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:32.918421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:32.924602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.925049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:32.925799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:32.926233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.060925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:33.069288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57ca2c54-b7ba-4d88-af91-78e43b6f0717 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155549 May 23 16:13:33.069846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1530/3166] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:32 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:33.465540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:33.466153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:33.472570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.473017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.473828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.474300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.535482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:33.543866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:33.545368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:33.546511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2b1d30b6-64a8-4059-b3bd-8eb85e27a9d5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084987 May 23 16:13:33.547066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1637/3167] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:33.700900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:33.701516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:33.707995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.708456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.709208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.709641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.769215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:33.777107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52438e14-3339-46a3-8f7f-012bd86da33b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080029 May 23 16:13:33.777683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1531/3168] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:33.905685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:33.906218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:33.912746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.913181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:33.913928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:33.914356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.046597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.054513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a61d07c2-9747-45c1-8cf1-84ac18ba85c6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152745 May 23 16:13:34.055066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1638/3169] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:34.086257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:34.086939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:34.092925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.093376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.094131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.094595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.233044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.241534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f2347e5a-81e6-4c48-90af-c52ff8d9c409 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159368 May 23 16:13:34.241861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1532/3170] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:34.564932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:34.565551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:34.572099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.572555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.573310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.573745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.633525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.641831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.643369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.644804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6c58dd6c-a7c1-4c36-8945-a57381a61a13 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083810 May 23 16:13:34.645359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1639/3171] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:34.793433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:34.794050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:34.800523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.800976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.801726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:34.802160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:34.862522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:34.870486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5351b19-2a59-4d1a-ac17-8fe376c8f4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080881 May 23 16:13:34.871050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1533/3172] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:35.071425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:35.072119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:35.078240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.078690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.079466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.079899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.211168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.219034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-607ac844-daef-4205-a755-2b7d5e2672ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.151567 May 23 16:13:35.219603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1640/3173] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:35.257568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:35.258278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:35.264411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.264857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.265603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.266035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.389984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.398259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-115e785b-f1b8-4f65-995d-0416c7ed26a4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.144506 May 23 16:13:35.398821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1534/3174] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:35 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:35.663152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:35.663761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:35.670073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.670523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.671301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.671732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.731307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.739546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.741033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.742163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-480f3634-18c8-4b1f-a9e9-0a672e8a2c10 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083555 May 23 16:13:35.742708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1641/3175] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:35.887064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:35.887700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:35.894536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.894982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.896355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:35.896789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:35.959198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:35.967036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8a96edb0-a069-4cc7-b893-6e47810a104a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083778 May 23 16:13:35.967618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1535/3176] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:36.238952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:36.240361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:36.247879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.248434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.249260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.249686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.377719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:36.385674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-759d0686-6bf9-44b4-9113-f19c9870266f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.151659 May 23 16:13:36.387178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1642/3177] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:36.415768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:36.416430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:36.422615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.423063 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.423877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.424310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.548575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:36.556413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c564dce0-f6a5-4734-b69b-79bc7c750054 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145120 May 23 16:13:36.556974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1536/3178] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:36.760214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:36.760747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:36.766867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.767338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.768092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.768522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.831411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:36.840231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:36.841751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:36.842886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8002fe22-5bfc-4c15-b54a-e0b078a05d24 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086656 May 23 16:13:36.843475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1643/3179] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:36.983607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:36.984257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:36.990798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.991299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:36.992059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:36.992495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.052275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.060136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-929d2212-cc81-45c4-9875-4f57a6c71da9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080441 May 23 16:13:37.060723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1537/3180] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:37.403802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:37.404353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:37.411050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.411534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.412303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.412743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.537135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.545064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3be16558-c917-4eef-b16f-f14f243b81a9 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.145208 May 23 16:13:37.545644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1644/3181] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:37.573927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:37.574542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:37.580579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.581031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.581786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.582224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.710897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.718741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-953eb2ab-0064-4e04-a7eb-7c937a1cc6ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149339 May 23 16:13:37.719329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1538/3182] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:37.860548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:37.861206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:37.867226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.867681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.868438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:37.868878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:37.928392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.936931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.938416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:37.939583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7b2750b8-3895-424f-9811-2387b755379b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.082950 May 23 16:13:37.940146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1645/3183] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:38.077374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:38.078024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:38.084563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.085612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.086379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.086822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.145789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:38.154247 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b774ecd2-9aae-4de0-9e4f-b1a03e11d694 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080762 May 23 16:13:38.154815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1539/3184] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:38.563168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:38.563714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:38.570611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.571053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.571820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.572293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.694908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:38.702652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-37082124-2a4f-4070-9f5f-1a45443a6a3c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.143423 May 23 16:13:38.703230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1646/3185] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:38.736423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:38.737052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:38.743337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.743794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.744554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.744993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.872617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:38.880691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7b651fa-a1ee-4336-85ae-99730cf43972 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148067 May 23 16:13:38.902293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1540/3186] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:38.957538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:38.958154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:38.964174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.964625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:38.965374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:38.965815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.025624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:39.033862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:39.035386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:39.036526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f910a7a8-9e42-498d-aed9-241fe612ff7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.082918 May 23 16:13:39.037080 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1647/3187] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:38 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:39.170888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:39.171469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:39.178391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.179360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.180170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.180614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.248574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:39.256479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-df963c8e-7729-4790-827e-b19dd1322c2d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089477 May 23 16:13:39.257212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1541/3188] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:39.720091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:39.720703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:39.727713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.728156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.728900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.729330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.746607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:39.858499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:39.866698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-919a67db-f4c9-4c74-b3c3-142fec7373d7 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.150415 May 23 16:13:39.867357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1648/3189] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:39.918300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:39.918941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:39.924997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.925446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:39.926194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:39.926629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.054431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:40.055188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:40.058413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:40.061562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.062085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.062939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.063481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.066244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a02a274-9a3e-4b79-a3ab-d49ebccce882 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151882 May 23 16:13:40.066925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1542/3190] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:39 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:40.123379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:40.131602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:40.133649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:40.134781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5c65f694-7503-4d46-9a68-ac13f23fa8ed tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084874 May 23 16:13:40.135359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1649/3191] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:40.274759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:40.275336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:40.281708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.282155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.282933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.283400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.346375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:40.354154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2456795d-4b4e-47b6-b86a-282f5fc3cac8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083904 May 23 16:13:40.354758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1543/3192] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:40.883846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:40.884500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:40.890975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.891461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:40.892253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:40.892686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.016217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.024327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dd2912a9-b10f-4c4a-9b18-e44b414ab9a6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144321 May 23 16:13:41.024890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1650/3193] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:41.083816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:41.084441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:41.090428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.090876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.091657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.092092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.095925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:41.153091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:41.153626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:41.159515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.159965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.160723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.161153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.232485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.241030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.242523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.243714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-326dda11-1c8d-40b5-92fa-1ee1dad99c9d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.094591 May 23 16:13:41.244263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1651/3194] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:41.249315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.257230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61271476-1f5e-4b86-b411-60243388754a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.177249 May 23 16:13:41.257779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1544/3195] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:41.370748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:41.371482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:41.377390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.377831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.378578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:41.379007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:41.439138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:41.446901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7df56889-1341-45e5-9b7b-25d54db7a33c tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079959 May 23 16:13:41.447485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1652/3196] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:42.044178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:42.044842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:42.050861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.051633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.052177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.052623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.175892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.183824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e154c50-4771-4d85-9952-5226a048cf80 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147163 May 23 16:13:42.184395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1545/3197] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:42.262098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:42.262737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:42.270594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.271042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.271832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.272265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.274663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:42.275337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:42.281291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.281740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.282502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.282940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.345728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.353977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.355501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.356641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3000b3aa-804d-4f30-8627-adc3105f5aea tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.099058 May 23 16:13:42.357985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1653/3198] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:42.426476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.434947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-df11320b-fa76-4cdf-b568-72ae184989d2 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.164923 May 23 16:13:42.435537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1546/3199] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:42.463424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:42.464106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:42.470148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.470600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.471387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:42.471830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:42.533049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:42.540900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ac4abd07-dc4d-43a6-8f15-0b93e1fa9a07 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081300 May 23 16:13:42.541462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1654/3200] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:43.201150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:43.201722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:43.208156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.208605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.209360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.209793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.338204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.346011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4b67792-6360-472a-bafa-0af60dd41801 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148898 May 23 16:13:43.346563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1547/3201] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:43.375599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:43.376204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:43.383027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.383502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.384244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.384670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.444047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.451988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:43.452548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:43.452831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.454323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.455481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e73cb1f0-78a0-4f27-946c-3c60e658bbb7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083664 May 23 16:13:43.456033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1655/3202] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:43.458636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.459123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.459900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.460340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.559706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:43.561146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:43.567395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.580569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.580569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:43.580569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:43.589306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.597707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fa2282f2-8f82-49c3-802d-a787bc56edfa tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149523 May 23 16:13:43.598284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1548/3203] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:43.632788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:43.640747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-df1732c2-5c6f-4684-a7bf-2ea1a045f6a4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086905 May 23 16:13:43.641312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1656/3204] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:44.362750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:44.363398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:44.372276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.372729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.373486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.373924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.473985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:44.474604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:44.485202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.485957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.486492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.486935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.524959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.533946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f900800b-af42-4aa1-9e49-ca7e4c1f57a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.175013 May 23 16:13:44.534533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1549/3205] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:44.557327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.566005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.567526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.568673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a4adc7f-fb12-430d-998d-2ea1972daec4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.098767 May 23 16:13:44.569228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1657/3206] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:44.614986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:44.615623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:44.622276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.622719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.623501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.623936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.657711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:44.658322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:44.666378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.666819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.667620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:44.668058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:44.743786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.751876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-300cff5a-6627-42b3-97d5-d48bc2202abe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.098010 May 23 16:13:44.752457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1658/3207] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:44.779992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:44.791648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-578b0bab-e203-4489-8ef9-c8c8825bfd7c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.180569 May 23 16:13:44.792275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1550/3208] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:45.551437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:45.552054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:45.559365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.559812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.560558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.560991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.592584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:45.593215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:45.600123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.601085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.601503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.601884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.674932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.684284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.685884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.687037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6bacf60d-8b82-4052-8e0c-7d56a25d7cff tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.099172 May 23 16:13:45.687619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1551/3209] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:13:45.709366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.717239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8510f54c-6160-454b-a674-34fe0cba4749 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.169719 May 23 16:13:45.717804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1659/3210] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:45.774021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:45.774636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:45.780841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.781291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.782038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.782471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.809698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:45.810242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:45.816271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.816723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.817483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:45.817974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:45.846245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.854853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52652f03-0966-4ba5-81a4-a7f4da7802ca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085150 May 23 16:13:45.855451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1552/3211] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:45.946128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:45.954061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-051403c6-9859-4677-8f00-ba33f65b35a0 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148319 May 23 16:13:45.954629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1660/3212] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:45 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:46.704538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:46.705069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:46.712120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.712563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.713311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.713744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.734628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:46.735356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:46.741778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.742233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.742999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.743463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.774456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:46.782878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:46.784419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:46.785559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7c13d702-5eba-4005-b434-59a5b92fd3c4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085040 May 23 16:13:46.786124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1553/3213] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:46.872201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:46.872378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:46.872972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:46.878881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.879364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.880121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.880231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c3909943-29a6-49ca-b0c6-effff4b72f79 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149355 May 23 16:13:46.880558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.888402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1661/3214] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:46 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:46.940840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:46.949317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-07e53c50-dbef-4873-bfc9-f6ade3ac1b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080835 May 23 16:13:46.949881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1554/3215] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:46.970880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:46.971457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:46.977329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.977772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:46.978522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:46.978953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.102999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:47.110768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-14473a81-defb-4b58-9bde-1b1686ba6c08 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.143765 May 23 16:13:47.111373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1662/3216] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:47.803375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:47.803925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:47.810563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.811011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.811798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.812237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.877115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:47.886054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:47.887596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:47.888749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-000d85e5-9f0a-4847-a9f3-34b2b016ef11 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089378 May 23 16:13:47.889310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1555/3217] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:47.897833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:47.899022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:47.905453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.905895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.906658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.907073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.966525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:47.967159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:47.973014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.973461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:47.974216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:47.974651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:48.044430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.052349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f4bdba25-ca90-41f8-b866-b56b37788926 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089862 May 23 16:13:48.053678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1556/3218] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:47 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:48.055475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.066372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c8d24f23-7f87-4ec6-8ca5-989b4b57c138 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.172145 May 23 16:13:48.067450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1663/3219] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:48.128412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:48.129023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:48.135624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:48.136076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:48.136829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:48.137267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:48.273820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.281717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9ecfd27e-540e-490f-a3cd-44b0eec5f577 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157116 May 23 16:13:48.282280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1557/3220] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:48.907463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:48.908094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:48.915239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:48.915686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:48.916431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:48.916859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:48.977563 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.985986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.987663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:48.988659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20485b0b-0775-4107-9d0d-86ffb7dbbee8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085097 May 23 16:13:48.989218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1664/3221] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:48 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:49.068859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:49.069506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:49.076072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.076544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.077294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.077733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.087065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:49.087702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:49.093614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.094057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.094899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.095357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.100300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:49.140654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:49.149598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2fa292c7-547a-4f33-8bf9-2e81c357e030 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084561 May 23 16:13:49.150183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1558/3222] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:49.238108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:49.246404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c561aa36-0eb2-4dcf-8277-e6a6a1277f8e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.163595 May 23 16:13:49.246976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1665/3223] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:49.298847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:49.299470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:49.305521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.305973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.306722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:49.307186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:49.430474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:49.438385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c113dc70-7050-420e-aae8-19383f93127f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.143370 May 23 16:13:49.438881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1559/3224] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:50.006578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:50.007817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:50.014371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.014816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.015598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.016031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.076204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.084400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.086088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.087247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a8f4516-1193-40d0-b546-8af8a21a8e5d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084580 May 23 16:13:50.087831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1666/3225] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:50.167362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:50.167986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:50.174443 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.180194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.180194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.180194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.242409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.245790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:50.250749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1b239497-66e9-409e-b8b4-aeef0d85612e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087677 May 23 16:13:50.252253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1560/3226] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:50.263342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:50.263958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:50.270273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.270717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.271522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.271935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.399871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.407598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f5bf008-5ba5-4bd0-aecb-5c82a2fac240 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148165 May 23 16:13:50.408214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1667/3227] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:50.455851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:50.456459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:50.462832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.463310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.464064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:50.464496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:50.589033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:50.597066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5682a13-e977-47d9-8b0a-c2deaa2d8fa5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.144997 May 23 16:13:50.597634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1561/3228] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:51.108202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:51.108813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:51.115217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.115663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.116403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.116830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.180178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.188397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.189893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.191026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-09c49c65-1896-40ed-ba1e-1a29564cac50 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086778 May 23 16:13:51.191608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1668/3229] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:51.267336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:51.267953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:51.275446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.275898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.276667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.277088 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.425433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:51.426062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:51.432784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.433225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.433993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.434421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.578996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.588130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-16f36371-c0be-4be4-8304-39d5e01091a5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.324628 May 23 16:13:51.588774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1562/3230] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:51.613635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:51.614245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:51.620156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.620586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.621337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:51.621769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:51.680230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.688229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fda43f10-62a0-4f11-b333-7564c4747648 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.266790 May 23 16:13:51.688805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1669/3231] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:51.761469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:51.769545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57791e5c-5c86-4c35-84a4-b7a2119cad53 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159522 May 23 16:13:51.770110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1563/3232] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:51 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:52.209853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:52.210450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:52.216949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.217400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.218150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.218587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.281161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.289510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.290995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.292161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-27e42d9d-7d97-4b9b-a1a1-305f3a5a31e6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086278 May 23 16:13:52.292710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1670/3233] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:52.604859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:52.605484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:52.612082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.612536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.613301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.613745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.674502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.683003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ce3911e-2a2e-408d-b31f-6a019318207f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082555 May 23 16:13:52.683816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1564/3234] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:52.705169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:52.705783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:52.711999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.712449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.713210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.713653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.787164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:52.787796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:52.794459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.794909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.795708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:52.796143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:52.867292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.875261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-be5d91da-a698-43e0-a2c0-f50cb92f6186 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.173820 May 23 16:13:52.875837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1671/3235] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:52.930653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:52.938852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6fcd25ea-1d23-4d7c-8150-061fce3a03da tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155463 May 23 16:13:52.939589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1565/3236] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:52 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:53.310475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:53.311081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:53.317493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.317939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.318684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.319137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.379160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:53.387258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:53.388754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:53.389884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4c33192f-21e0-4884-bcf1-7c377b641bc5 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083365 May 23 16:13:53.390463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1672/3237] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:53.699840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:53.700364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:53.706818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.707302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.708054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.708486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.771252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:53.779537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-29f66873-dab4-4308-a9fe-104b2c58f884 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083606 May 23 16:13:53.780100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1566/3238] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:53.892633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:53.893249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:53.900114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.900568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.901320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.901753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.957427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:53.957962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:53.965094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.965538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:53.966427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:53.966890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:54.065848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.074384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8c828cc4-142d-4f96-946a-463cae4171de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.185556 May 23 16:13:54.074974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1673/3239] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:54.109991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.117825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a9c2f27-590c-44b1-b704-0bc76a1e6821 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.164330 May 23 16:13:54.118565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1567/3240] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:54.408186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:54.408803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:54.415337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:54.415789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:54.416542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:54.416976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:54.480500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.488889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.490331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.491998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-facaa29c-481d-4554-95a8-7a402a97550c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087236 May 23 16:13:54.492556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1674/3241] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:54.796775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:54.797325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:54.803937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:54.804389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:54.805154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:54.805638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:54.865888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:54.873805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a01a0042-f4b5-4a99-9a48-b4a3be4316c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081006 May 23 16:13:54.874366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1568/3242] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:55.091522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:55.092066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:55.098026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.098466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.099246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.099705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.142669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:55.143429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:55.150188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.150636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.151416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.151849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.280032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.288045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-86df7eea-5fb2-4507-821a-1187f55977f6 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.200436 May 23 16:13:55.288612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1675/3243] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:55 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:55.308481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.316387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-90b44e0c-f724-45d0-8461-3a8d3f930ac1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.177629 May 23 16:13:55.316949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1569/3244] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:55.509701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:55.510322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:55.517153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.517610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.518371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.518809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.579067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.587445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.588992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.590139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3ced181d-7626-4565-992b-e030c3942327 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084423 May 23 16:13:55.590698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1676/3245] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:55.890480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:55.891011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:55.897289 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.897729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.898513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:55.898948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:55.958241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:55.966073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a34bdf58-271e-4365-8d15-397d8d862039 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079529 May 23 16:13:55.966633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1570/3246] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:56.304945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:56.306125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:56.308058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:56.312809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.313249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.313995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.314423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.333868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:56.334521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:56.337848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:56.341302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.341752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.342515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.342955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.512106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:56.516018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:56.521022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-025bf287-d853-481a-a737-8af1294afb58 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.219865 May 23 16:13:56.521915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1677/3247] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:56.525041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83e1faef-e89f-4d26-b828-eba3194ed73e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.194931 May 23 16:13:56.525686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1571/3248] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:56 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:56.607696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:56.608296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:56.614420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.614871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.615666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.616103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.677907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:56.686169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:56.687696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:56.688867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-145d804d-851b-490d-8dc5-820614e9b926 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084940 May 23 16:13:56.689422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1678/3249] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:56.982779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:56.983426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:56.989510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.989960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:56.990714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:56.991176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.053660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.061550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe4e9b3e-cb0f-458c-ab53-63d44fefa235 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082584 May 23 16:13:57.062111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1572/3250] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:57.539497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:57.540054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:57.541507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:57.542125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:57.546467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.546920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.547712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.548149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.548525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.548971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.549721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.550184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.737307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.741277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.745917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4303ee09-1de0-45ad-9131-7106bd28de0b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.210371 May 23 16:13:57.746490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1679/3251] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:57.749863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a59c50ff-ce13-4510-a3e7-5654d159b9ed tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212202 May 23 16:13:57.750554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1573/3252] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:57.756169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:57.756833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:57.762802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.763294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.764056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:57.764495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:57.824555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.832838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.834330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:57.835497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6bb862ce-017f-4fdf-b76c-a89ebb659a3e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088062 May 23 16:13:57.836052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1680/3253] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:58.078305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:58.078844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:58.085684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.086134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.086887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.087354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.146924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:58.159883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d1fd25a5-db20-479f-92ce-7b055ab50f25 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085441 May 23 16:13:58.160464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1574/3254] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:58 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:58.762772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:58.763405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:58.767414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:58.768032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:58.770201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.770653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.771448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.771887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.774866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.775341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.776108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.776526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.958666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:58.962145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:58.962690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:58.967008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e3d4c8e-c24c-4e78-ae18-c4fe9bb08f65 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.208169 May 23 16:13:58.967619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1681/3255] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:58.970799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0576fdd2-e342-4d19-8854-66e32c841d0f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.207678 May 23 16:13:58.972642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1575/3256] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:13:58.972781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:58.972781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:58.978450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.978911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:58.979697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:58.980136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:59.040721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:59.049005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:59.050526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:59.051731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24b99526-934f-4a4c-bc0c-3781c4c7a418 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083189 May 23 16:13:59.052289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1682/3257] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:13:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:13:59.179585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:59.180203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:59.186234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:59.186686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:59.187473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:59.187910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:59.251960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:13:59.259910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6594824b-ea15-42e1-adef-9630406fe774 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084760 May 23 16:13:59.260475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1576/3258] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:13:59.984233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:59.984830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:13:59.991691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:59.992167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:59.992948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:13:59.993283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:13:59.993409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:13:59.993903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:00.000212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.000663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.001421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.001859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.020359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:00.189662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.193670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.198872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-47481b4e-fcad-48a4-ade1-021d7045078f tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.218445 May 23 16:14:00.199488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1683/3259] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:00.202406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6eed07d9-68ff-4eb4-8f24-8b3530cc62dc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.212883 May 23 16:14:00.203117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1577/3260] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:13:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:00.204730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:00.205347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:00.211750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.212197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.212955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.213415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.275255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.277107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:00.277725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:00.283654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.283808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.284112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.284871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:00.285222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.285483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:00.286341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-35470967-92fc-4672-afae-f6d6756d639d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085945 May 23 16:14:00.286893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1684/3261] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:00.345067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:00.352973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aa1e0e4f-0206-4d5e-b13b-478a8983f12b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080231 May 23 16:14:00.353483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1578/3262] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:01.216423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:01.216984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:01.223547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.223996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.224747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.225177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.334154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:01.334778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:01.341526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.342105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.342875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.343346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.352831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.360882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e8d0058-877a-4fd3-bcad-363db45d45bb tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148373 May 23 16:14:01.361919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1685/3263] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:01.473768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.481782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-04b1016d-0ce9-4ac2-ab13-ba9f0435cfc4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.265836 May 23 16:14:01.482346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1579/3264] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:01.484543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:01.485157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:01.491324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.491776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.492619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.493095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.557374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.566769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.568303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.569441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d68c8f64-7ae7-4175-a4d9-c65a237c7e3d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.207092 May 23 16:14:01.570004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1686/3265] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:01.611639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:01.612263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:01.618222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.618670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.619469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:01.619912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:01.680921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:01.688789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6451cb8c-6e9f-4772-a3a3-6467bada1257 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.205508 May 23 16:14:01.689355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1580/3266] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:02.377810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:02.378439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:02.384903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.385347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.386091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.386521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.499695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:02.500260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:02.506625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.507073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.507861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.508291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.514206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.522326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a01e725b-9226-4858-a8c6-f2c5e6cd4e0d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148343 May 23 16:14:02.522891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1687/3267] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:02 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:02.588678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:02.589293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:02.595338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.595781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.596564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.596996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.647028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.655535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a70fe72-ee36-48e7-9dee-37a97e355348 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159793 May 23 16:14:02.656105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1581/3268] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:02 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:02.662880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.672073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.673588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.674738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e06dbe8-a413-42e6-a203-7a029abc9285 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.090645 May 23 16:14:02.675320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1688/3269] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:02.705553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:02.706196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:02.712744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.713197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.713955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:02.714392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:02.777375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:02.785074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a795a45-4c98-499d-bea9-57bcf826d5e9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083416 May 23 16:14:02.785644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1582/3270] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:03.540023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:03.540587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:03.547116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.547569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.548323 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.548761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.673255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:03.673867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:03.681013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.681464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.682231 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.682755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.685320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.693517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-44871d2f-783b-4839-ba6c-3b0e65145593 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.157481 May 23 16:14:03.694084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1689/3271] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:03.698574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:03.699200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:03.704806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.705248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.706000 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.706431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.779040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.787371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.788866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.790591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e408cf25-54ed-404e-a810-22835693c49b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.095544 May 23 16:14:03.791100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1690/3272] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:03.802010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:03.802629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:03.809369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.809634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.810399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:03.810838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:03.835975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.844048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-88361853-5949-4879-b6e3-6f204d46fbb1 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.174654 May 23 16:14:03.844621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1583/3273] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:03.870795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:03.878619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-122724da-5a25-43c2-897c-71e9eeff5461 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080301 May 23 16:14:03.879206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1691/3274] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:04.710653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:04.711233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:04.718379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.718836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.719638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.720084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.808839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:04.809451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:04.816073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.816529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.817293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.817732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.847273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:04.855343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e04e0d13-cef7-42eb-9ff1-d5e5aa3f6978 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148547 May 23 16:14:04.856666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1584/3275] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:04 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:04.861178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:04.861795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:04.867743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.868715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.869476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.869912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.881452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:04.890871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:04.892459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:04.893747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4d9e238-07fc-42c8-afde-c1cb0b9ed23c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089278 May 23 16:14:04.894765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1692/3276] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:04 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:04.898968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:04.899612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:04.905534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.905987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.906781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:04.907255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:04.976569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:04.984500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9c9e8f5-78d6-4168-bf5e-f1b506c85e7f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088956 May 23 16:14:04.985077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1693/3277] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:04 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:05.019018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:05.027205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3f301f37-2e68-44a7-b31e-13523898f3bf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.169565 May 23 16:14:05.027776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1585/3278] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:04 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:05.872647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:05.873271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:05.879390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:05.879843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:05.880602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:05.881038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:05.911899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:05.912514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:05.919621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:05.920053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:05.920819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:05.921254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:06.003295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.012581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.014083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.015270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-30366e2e-8b56-4cee-b98c-b05c987c9b0f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.107217 May 23 16:14:06.016093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1586/3279] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:05 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:06.020797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:06.021652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:06.028990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:06.029817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:06.030744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:06.031212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:06.033131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.041994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-32107c66-511c-40d7-9f68-4605013a36d1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.172939 May 23 16:14:06.042667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1694/3280] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:05 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:06.047184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:06.047873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:06.054326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:06.054778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:06.055584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:06.056084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:06.094445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.102955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-339908e0-7532-4cae-9723-069b5e9461fc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085827 May 23 16:14:06.103689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1587/3281] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:06 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:06.194800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:06.202790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-13f3761b-542e-477b-a053-f8ad58b19328 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.159091 May 23 16:14:06.203394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1695/3282] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:06 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:07.036189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:07.036837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:07.043898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.044352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.045110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.045549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.059625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:07.060243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:07.066593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.067037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.067832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.068264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.110846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.120337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.121866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.123005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0a5fbe4f-3e47-4f30-8fcd-d145ed277490 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.090696 May 23 16:14:07.123599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1588/3283] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:07 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:07.127987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:07.128610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:07.134611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.135143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.136434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.136874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.206910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.214804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70ac58b0-f28a-445a-a7d8-8637f1912c95 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090237 May 23 16:14:07.215402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1589/3284] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:07 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:07.219809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:07.220424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:07.226235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.226692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.227488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:07.227933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:07.231143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.241538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff070f5e-865e-4c98-8e44-780a685da055 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.186171 May 23 16:14:07.242122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1696/3285] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:07 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:07.361044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:07.368899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-03125f71-f35b-4679-aba0-57f40d738719 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152562 May 23 16:14:07.370000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1590/3286] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:07 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:08.141402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:08.142014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:08.148780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.149223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.149966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.150393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.218312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.226823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.228351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.229560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3dbc6e06-d49b-4cef-88a7-0231c200ab4c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.092058 May 23 16:14:08.230265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1697/3287] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:08 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:08.232227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:08.232853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:08.240979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.241314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.242089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.242522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.259027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:08.259840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:08.266517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.266970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.267773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.268213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.286591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:08.305570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.313809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c1c9337b-d7ca-47f3-bb86-45bb826cd988 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085878 May 23 16:14:08.314379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1591/3288] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:08 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:08.385606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:08.386225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:08.392239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.392626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.393375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:08.393803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:08.401861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.409823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ed33507-804f-42eb-a7e9-8f5e956dac44 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.154554 May 23 16:14:08.410404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1698/3289] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:08 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:08.520100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:08.528522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-318e4351-c63a-4c97-ad4c-493f3360dd16 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146711 May 23 16:14:08.529083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1592/3290] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:08 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:09.248247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:09.248869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:09.255434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.255883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.256649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.257089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.318347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.327414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.328933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.330085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-757d3150-98e0-404f-99fa-6f321c4bf415 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086642 May 23 16:14:09.330639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1699/3291] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:09 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:09.331234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:09.331942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:09.338379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.338826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.339617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.340050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.403358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.411234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e812a49c-aa57-4cc9-9608-7697f164c0cb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083859 May 23 16:14:09.411802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1593/3292] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:09 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:09.427143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:09.427800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:09.434332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.434786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.435584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.436028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.545590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:09.546208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:09.552174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.552627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.553388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:09.553858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:09.558336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:09.564420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.572506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-febdbd91-a3ee-4e45-925e-82ecf4a45f49 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149117 May 23 16:14:09.573968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1700/3293] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:09 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:09.680914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:09.689378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4a6a4b33-fc87-43d4-ba5f-1aad6d545eb4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147660 May 23 16:14:09.689944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1594/3294] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:09 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:10.347839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:10.348445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:10.354856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.355335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.356085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.356513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.420408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.427002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:10.427781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:10.429179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.430696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.431870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d2c92220-7b42-4bf1-b17b-2b8d532c443f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087925 May 23 16:14:10.432435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1701/3295] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:10 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:10.434307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.434762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.435560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.435997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.497431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.505226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c5d8cf3-59cc-49f1-af60-6fcf1345d56f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082067 May 23 16:14:10.505790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1595/3296] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:10 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:10.590177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:10.590821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:10.596766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.597212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.597969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.598401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.705756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:10.706503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:10.712500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.712958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.713729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:10.714165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:10.722734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.730811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-64d01e7a-98f2-4899-923d-93bced3414cd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.145049 May 23 16:14:10.731426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1702/3297] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:10 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:10.842686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:10.850523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-898e4a82-a3ca-44f7-93ab-b0fc3ce8f898 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148638 May 23 16:14:10.851115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1596/3298] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:10 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:11.450101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:11.450716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:11.457213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.457664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.458414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.458850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.520429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:11.522704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:11.523355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:11.529822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:11.530332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.530787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.531463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:11.531624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.532014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.532544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70ae1bfe-b4cd-45c2-a36b-871fab1c1e93 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086375 May 23 16:14:11.533089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1703/3299] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:11 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:11.593933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:11.602399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b5dc0df6-6f73-44b2-8830-9872ba31d4bd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083297 May 23 16:14:11.602524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1597/3300] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:11 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:11.748021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:11.748630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:11.754720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.755194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.755946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.756379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.866945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:11.867581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:11.874017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.874480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.875282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:11.875729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:11.882723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:11.891257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d8bfcadd-b267-46e0-acd5-508f6b404d35 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147146 May 23 16:14:11.891840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1704/3301] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:11 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:12.000531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:12.008402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4e12f299-ef57-4a8f-81c0-37ff53f1515c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145295 May 23 16:14:12.008976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1598/3302] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:11 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:12.551446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:12.551997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:12.558885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.559357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:12.560113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.560547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:12.619262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:12.620470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:12.621278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:12.626982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.627495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:12.628262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.628697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:12.629698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:12.631220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:12.632365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b3b0d06-5a53-451c-b9f4-7b55d1e11c6c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084967 May 23 16:14:12.633113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1705/3303] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:12 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:12.688974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:12.696939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cb030690-0bc9-4767-8c13-3f227e87c097 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081627 May 23 16:14:12.697504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1599/3304] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:12 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:12.909252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:12.909822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:12.915822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.916270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:12.917513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:12.917946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.026696 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:13.027385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:13.033441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.033896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.034665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.035152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.049604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.057516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77c9bfd5-d877-4320-b03c-50a2b88629a4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152200 May 23 16:14:13.058085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1706/3305] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:12 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:13.162123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.170676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b531a81-8a39-4aef-b88f-365d749f62ca tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148445 May 23 16:14:13.171260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1600/3306] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:13 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:13.650283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:13.651546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:13.658114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.662950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.662950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.662950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.713804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:13.714413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:13.721337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.721784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.722547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:13.722977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:13.733372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.741969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.743506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.745199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b55ee67b-de5b-41f0-bea8-dc49788de5d1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.098269 May 23 16:14:13.745960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1707/3307] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:13 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:13.785629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:13.793583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4c12746d-38c3-4e0b-8649-17a5e90f4e40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083750 May 23 16:14:13.794261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1601/3308] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:13 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:14.080777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:14.081431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:14.087510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.087965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.088727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.089170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.188332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:14.188941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:14.194849 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.195326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.196080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.196512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.218870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.226919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-040f5379-99b4-4e59-85fe-3e8d913e55e3 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.150119 May 23 16:14:14.227524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1708/3309] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:14 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:14.327312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.335182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a1a3413-9344-48bb-95d5-5f806abe16af tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.150619 May 23 16:14:14.335687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1602/3310] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:14 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:14.763201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:14.763821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:14.770633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.771090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.771868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.772309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.809621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:14.810232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:14.816710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.817154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.817915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:14.818345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:14.833564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.842747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.844409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.845641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-544264fb-619e-45d8-850f-f84be2ed238a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086444 May 23 16:14:14.846289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1709/3311] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:14 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:14.880246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:14.888580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60f8e416-1cbf-46ad-84f4-0693cd5f577b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082818 May 23 16:14:14.889139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1603/3312] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:14 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:15.245003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:15.245648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:15.251670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.252121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.252872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.253308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.352438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:15.353048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:15.359253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.359702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.360458 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.360883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.381039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.393522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af5592ff-6be7-4dc2-b885-7f6093dce963 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152872 May 23 16:14:15.394114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1710/3313] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:15 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:15.489045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.497453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c72eb4b6-baae-4dd8-a65e-b1661980af76 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149068 May 23 16:14:15.498021 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1604/3314] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:15 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:15.863525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:15.864053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:15.871051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.871535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.872292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.872729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.905678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:15.906394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:15.913112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.913566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.914667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:15.915242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:15.935824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.944469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.946118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.947302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-71f76936-3016-42cd-96b9-011b984052a4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087777 May 23 16:14:15.947901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1711/3315] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:15 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:15.979728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:15.987657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3598fb65-545a-406e-8434-c1c3b822168d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085852 May 23 16:14:15.988228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1605/3316] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:15 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:16.411711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:16.412336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:16.418421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.418971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:16.419763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.420208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:16.514181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:16.514796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:16.521140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.521594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:16.522356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.522790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:16.547368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:16.555439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d986ed77-77e1-4223-b1cb-b0d435444450 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147545 May 23 16:14:16.556152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1712/3317] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:16 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:16.646630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:16.655077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d41d2ce0-88f5-4652-82d5-4d85ad2e6ed9 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145295 May 23 16:14:16.655658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1606/3318] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:16 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:16.965519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:16.966142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:16.972720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.973178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:16.973941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:16.974384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.004302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:17.004919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:17.012174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.012634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.013406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.013852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.040885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.049984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.051506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.052651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e797850-475c-4b3a-8f72-8ec05a2bb259 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.091493 May 23 16:14:17.053208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1713/3319] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:16 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:17.075812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.083687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06f6a48a-f3e7-4010-8c9e-fa146ca0771f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083194 May 23 16:14:17.084244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1607/3320] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:16 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:17.572627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:17.573183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:17.579274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.579728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.580486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.580924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.672952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:17.673549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:17.679598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.680042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.680793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:17.681225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:17.706542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.715150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03e9cd76-502c-476b-b7c0-43b7ad56230e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146453 May 23 16:14:17.716003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1714/3321] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:17 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:17.809931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:17.817807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e648ce08-3dcf-4699-b063-edaa9b09e5b5 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148671 May 23 16:14:17.818364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1608/3322] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:17 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:18.071006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:18.071643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:18.078495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.078959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.079738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.080167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.084014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:18.100379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:18.102345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:18.107518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.107973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.108724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.109163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.142706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.152145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.153759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.154915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0b37f73-350b-493f-bbe3-1097596d93e8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087826 May 23 16:14:18.155539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1715/3323] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:18 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:18.170894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.178812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f1780d94-9a78-4876-a09a-ae99ebee9b19 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082250 May 23 16:14:18.179397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1609/3324] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:18 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:18.732599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:18.733214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:18.739435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.739887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.740667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.741131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.835180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:18.835800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:18.841988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.846747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.846747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:18.846747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:18.869445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.878025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1914a295-cc13-40ce-8141-3d82e5b4a3de tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149787 May 23 16:14:18.878591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1716/3325] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:18 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:18.969620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:18.973015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:18.977819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3f417e68-7af4-4886-a829-5fe47674e4cb tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146485 May 23 16:14:18.979547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1610/3326] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:18 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:19.173556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:19.174176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:19.180546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.180989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.181734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.182219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.196474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:19.197090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:19.203813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.204259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.205009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.205438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.245044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:19.253691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:19.255220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:19.256361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-27bf56b0-8baa-47e1-9200-8fcf00a15792 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086812 May 23 16:14:19.256914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1717/3327] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:19 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:19.270404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:19.278771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b3cc92b-78e3-4a26-a387-8bbfb464ec53 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086474 May 23 16:14:19.279361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1611/3328] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:19 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:19.895406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:19.896016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:19.901938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.902381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.903145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:19.903580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:19.999784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:20.000415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:20.006351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.006808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.007617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.008061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.032256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.040153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-72331e8e-b687-48a0-b7c9-933bfb886f2e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148899 May 23 16:14:20.040713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1718/3329] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:19 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:20.139918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.148607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5e57e1f-90b4-4cd4-950b-77c50bc9f4bc tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152471 May 23 16:14:20.149251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1612/3330] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:19 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:20.274086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:20.274698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:20.281568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.282008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.282753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.283220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.295868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:20.296487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:20.302924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.303405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.304173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:20.304611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:20.345956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.355423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.356967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.358129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-01de8f35-8ce6-4e28-ab0d-89ffc4b27c79 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087953 May 23 16:14:20.358688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1719/3331] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:20 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:20.375580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:20.383494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c58e306b-d372-4d32-94fd-051db4dc8212 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.092001 May 23 16:14:20.384048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1613/3332] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:20 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:21.057105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:21.057712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:21.063760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.064202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.064951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.065379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.165494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:21.166106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:21.172186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.172919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.174227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.175071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.194173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.202687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a0ea0de-c332-4f0a-97fc-b5e99762fb45 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149403 May 23 16:14:21.203291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1720/3333] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:21 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:21.302025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.311011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a521313-4c99-44f7-8f91-c0b2c25a56f4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149267 May 23 16:14:21.311548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1614/3334] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:21 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:21.376320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:21.376931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:21.383289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.383733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.384482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.384933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.400616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:21.401242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:21.407348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.407801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.408574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:21.409013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:21.456519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.465780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.467360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.468548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6cf43180-69fe-447b-8c4f-cd97af9f8891 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.096077 May 23 16:14:21.469108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1721/3335] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:21 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:21.483432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:21.491365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-49e9ebe5-31df-4676-b615-6e4e5ac64049 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.094849 May 23 16:14:21.491932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1615/3336] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:21 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:22.219481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:22.220049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:22.226046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.226490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.227278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.227713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.328812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:22.329430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:22.336035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.336486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.337250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.337686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.363562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.372461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4076fd3e-8a8c-45b3-9fc8-cadc4b3a137a tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.157120 May 23 16:14:22.373043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1722/3337] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:22 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:22.479694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.487171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:22.487819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:22.487912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0911f95-dfee-4975-9e1b-c7be30624e44 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.163066 May 23 16:14:22.488447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1616/3338] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:22 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:22.494497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.494949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.495733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.496165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.507867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:22.508479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:22.514993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.515505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.516267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:22.516702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:22.560747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.569451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.571694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.573468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-077352d6-e7fc-48a2-b6a4-a67a6fe32d83 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.090426 May 23 16:14:22.573949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1723/3339] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:22 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:22.586611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:22.595802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd2cbaaf-e5aa-48ff-8efc-ee1a099f3261 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091577 May 23 16:14:22.596356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1617/3340] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:22 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:23.389274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:23.389884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:23.396225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.396670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.397413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.397844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.505290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:23.505831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:23.512879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.513326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.514089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.514524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.525550 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.533477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d16b50e1-bfbd-4a28-bdcb-8366e4716014 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148510 May 23 16:14:23.534045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1724/3341] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:23 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:23.591215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:23.591824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:23.597937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.598381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.599158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.599591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.650577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.658633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f45f30ad-b13a-471e-a228-53e4617f364c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157385 May 23 16:14:23.659232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1618/3342] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:23 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:23.663623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:23.664236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:23.666325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.670258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.670712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.671518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:23.671958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:23.675334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.676838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.678003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9b28a50d-62a4-4d3a-8f55-6233125a32c7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.090707 May 23 16:14:23.678715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1725/3343] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:23 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:23.731183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:23.739039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f9b91b48-9310-4d3b-bbdd-c9268cca9be6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.078872 May 23 16:14:23.739620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1619/3344] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:23 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:24.549986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:24.550596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:24.556993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.557446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.558189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.558619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.676019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:24.685313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:24.685313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.685313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.685313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.686071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.687688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.695820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aff9a1b8-1139-432d-a5ed-6ee58083ac02 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149640 May 23 16:14:24.696397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1726/3345] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:24 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:24.701436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:24.702621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:24.704779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:24.708152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.709582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.710346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.710783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.780704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.788937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.790439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.791625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de7bb65a-2160-41b5-8473-dc38739b8669 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.094319 May 23 16:14:24.792192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1727/3346] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:24 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:24.796552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:24.797163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:24.803060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.803540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.804306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:24.804743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:24.839143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.847598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a97aee0-0b8e-4ac9-a414-f7579ab63c59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.175435 May 23 16:14:24.848171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1620/3347] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:24 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:24.869828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:24.877785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b0060a25-1ee6-4032-be6d-3d40b22d95ea tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084630 May 23 16:14:24.878343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1728/3348] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:24 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:25.712787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:25.713406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:25.716229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:25.720973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.721420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.722167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.722596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.809632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:25.810249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:25.816625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.817078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.817841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.818279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.849102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:25.857904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-edf71e97-4ac6-4a59-8c1c-6d6701ea5fd1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149036 May 23 16:14:25.858485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1621/3349] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:25 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:25.872537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:25.873180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:25.878701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:25.879191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.879636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.880390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.880993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.887374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:25.888861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:25.890046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8e12368-d1a2-4fb8-8e79-f497fd4ae372 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084267 May 23 16:14:25.890552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1729/3350] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:25 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:25.895115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:25.895723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:25.901512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.901956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.903244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:25.903682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:25.990517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:25.998498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-21c990e6-90c0-4ac9-a392-8fa4777934a9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.106997 May 23 16:14:25.999305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1730/3351] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:25 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:26.035501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:26.043400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdcdcccf-a4ad-4ccb-86bd-f9013bfe5a82 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.174877 May 23 16:14:26.043952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1622/3352] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:25 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:26.875012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:26.875662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:26.882069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:26.882513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:26.883291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:26.883725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:26.908402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:26.909007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:26.915433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:26.915881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:26.916638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:26.917071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:26.996334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.004765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.006285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.007463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75fadc6e-5ea8-4362-a4af-1f9852966c7e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.102876 May 23 16:14:27.008024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1623/3353] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:26 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:27.014882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:27.015545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:27.021776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:27.022233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:27.023004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:27.023472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:27.034597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.038030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:27.043831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6a772fe2-5d7b-4c17-b7a1-9fa1d25d73ed tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.172855 May 23 16:14:27.044522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1731/3354] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:26 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:27.059421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:27.063167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:27.066187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:27.066634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:27.067427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:27.067862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:27.084740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.093155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-519ffe2b-e367-416b-800f-cee1b152ea7e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082911 May 23 16:14:27.093725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1624/3355] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:27 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:27.194031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:27.202587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3b0dbf10-9ede-4809-adb3-2df1cb2fca0d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146895 May 23 16:14:27.203177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1732/3356] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:27 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:28.026149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:28.026767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:28.033339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.033794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.034560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.035004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.061333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:28.062039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:28.068693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.069149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.069927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.070368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.099029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.107861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.109400 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.110566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-032424b3-f907-4c9f-92b7-1872b958f11a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088363 May 23 16:14:28.111155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1625/3357] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:28 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:28.116122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:28.116732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:28.123208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.123650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.124396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.124824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.193689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.202440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e54c14cf-38db-4eed-89de-22233b51e605 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090303 May 23 16:14:28.203016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1626/3358] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:28 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:28.219321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:28.219930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:28.223522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.225879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.226324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.227073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:28.227529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:28.232308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-57c2f7a2-fe48-4d1f-9356-019a3c24e0a0 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.175281 May 23 16:14:28.232898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1733/3359] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:28 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:28.246027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:28.364542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:28.373171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf2a8aac-b874-43b9-a454-f6dbebeed6a6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157499 May 23 16:14:28.373734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1627/3360] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:28 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:29.129131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:29.129726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:29.136227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.136675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.137483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.137918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.202711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.211409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.212914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.214072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d44dfd02-2d53-418e-b749-b04d54e5e72d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088890 May 23 16:14:29.214627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1734/3361] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:29 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:29.219852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:29.220542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:29.227317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.227833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.228554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.229029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.249376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:29.249993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:29.256089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.256546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.257315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.257755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.291875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.300036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f8b82796-036f-42e1-b95b-7019a88f1e74 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084641 May 23 16:14:29.300626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1628/3362] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:29 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:29.383671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.390285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:29.390905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:29.393079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3db793d3-c790-42b1-8c3f-ac9d84f82f92 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147464 May 23 16:14:29.393666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1735/3363] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:29 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:29.397326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.397779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.398555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:29.398998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:29.779008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:29.787321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a6c30932-1b17-4320-bee2-ef8ea03d7f3f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.401276 May 23 16:14:29.787898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1629/3364] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:29 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:30.233572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:30.234321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:30.246275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.246719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.247510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.247944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.319628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:30.320437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:30.321206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.327018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.327522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.328284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.328721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.329532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.331035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.332368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31046d0a-ffdf-437f-aa7f-2c3a693b1c22 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.102733 May 23 16:14:30.332940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1736/3365] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:30 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:30.394217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.401675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b426dec1-8cdf-4714-be5f-ffed12f9cf61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.086585 May 23 16:14:30.402245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1630/3366] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:30 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:30.409726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:30.410346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:30.416495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.416942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.417711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.418130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.559671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.567620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e65be27c-80db-46b2-b3f8-b7d71ce45c4c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.161831 May 23 16:14:30.568186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1737/3367] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:30 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:30.814061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:30.814594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:30.820737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.821189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.821944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:30.822380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:30.949192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:30.957213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-24536ef5-71a7-40f8-9916-e52c6bd2be65 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.147233 May 23 16:14:30.957807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1631/3368] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:30 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:31.350735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:31.351378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:31.357937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.358381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.359161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.359599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.421385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:31.421794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:31.423250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:31.428103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.428663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.429591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.430034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.431653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:31.433151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:31.434405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5dc3d271-04f5-46a2-bf2a-b43e787ed65c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087664 May 23 16:14:31.434973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1738/3369] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:31 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:31.491227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:31.499550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-98269569-ae61-4017-b3c0-bdbde4182706 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082670 May 23 16:14:31.502924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1632/3370] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:31 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:31.584222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:31.584873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:31.591630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.592085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.592962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.593403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.723467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:31.731989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7fd86da4-900a-4195-af9c-e126caef5120 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.151630 May 23 16:14:31.732745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1739/3371] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:31 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:31.974791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:31.975502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:31.981845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.982301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:31.983075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:31.983534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.116830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.125427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a891cff9-23e4-4267-84b9-fc862f0c6707 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.154470 May 23 16:14:32.126051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1633/3372] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:31 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:32.452594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:32.453303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:32.461284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.461735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.462507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.462943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.517833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:32.518383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:32.525759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.526214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.526979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.527995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.528980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.537280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.538819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.539996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0b6409b0-8bfb-434e-8236-94a1112c9e84 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.091352 May 23 16:14:32.540558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1740/3373] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:32 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:32.589042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.597570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e93031eb-9491-469e-a8da-ff1fc402e463 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083693 May 23 16:14:32.598143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1634/3374] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:32 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:32.749147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:32.749655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:32.755782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.756226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.756970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:32.757396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:32.895658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:32.903558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2af64dea-b0a7-4cbd-9a40-0c206e870b3d tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.158355 May 23 16:14:32.904120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1741/3375] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:32 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:33.144069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:33.144937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:33.151266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.151722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.152521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.153487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.285014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:33.292999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a3d38ff-06a0-4293-bec4-0d4a88f2c1ef tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152906 May 23 16:14:33.294144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1635/3376] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:33 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:33.559079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:33.559757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:33.566281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.566730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.567519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.567956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.615771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:33.616530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:33.623281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.623745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.624566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.625010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.632203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:33.641410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:33.642943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:33.644123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1c2783e-5cbc-40aa-8e19-9153ffe53c47 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089856 May 23 16:14:33.644947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1742/3377] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:33 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:33.686435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:33.698894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-80f2fa65-734f-499b-b789-eac071398023 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088449 May 23 16:14:33.700047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1636/3378] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:33 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:33.921058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:33.921596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:33.927618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.928067 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:33.928820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:33.929257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.065178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.073050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e67d15c-b2d2-4ba2-a75a-e3da1a77ed33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.155881 May 23 16:14:34.073633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1743/3379] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:33 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:34.309805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:34.310432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:34.316532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.316987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.317749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.318197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.447922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.457876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d51cbd72-588b-4139-beaa-dbe6d9bf9c40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151528 May 23 16:14:34.458955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1637/3380] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:34 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:34.664012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:34.664661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:34.671672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.672120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.672881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.673316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.716650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:34.717246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:34.723723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.724170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.724922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:34.725354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:34.737704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.746454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.747981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.749135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5743d9a1-2ee6-4da9-92e3-d375c9b5b27c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089063 May 23 16:14:34.749864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1744/3381] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:34 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:34.792883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:34.801305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17d4a61e-fac3-4ddc-9978-eabcc753cf99 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089012 May 23 16:14:34.801868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1638/3382] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:34 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:35.091429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:35.092057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:35.098149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.098605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.099399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.099840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.240276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.249771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e59db7d-f497-4fc9-9881-9433416deb33 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.162534 May 23 16:14:35.250741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1745/3383] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:35 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:35.476062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:35.476588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:35.483220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.483670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.484421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.484853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.625284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.633645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06ff05eb-2b96-4c75-97e7-c2a665527e03 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.161465 May 23 16:14:35.634234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1639/3384] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:35 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:35.768452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:35.769053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:35.775926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.776381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.777140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.778202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.817790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:35.818399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:35.827172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.827172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.827581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:35.827818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:35.849771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.858580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.860163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.861626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45835c36-8de2-41a2-8822-034eaafb2dc4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.097635 May 23 16:14:35.862225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1746/3385] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:35 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:35.892744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:35.901521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2887c8ce-14c6-4c96-ace6-b69b4af2eaa8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087642 May 23 16:14:35.902089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1640/3386] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:35 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:36.267303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:36.267926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:36.274341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.274813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.275604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.276050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.428558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:36.436655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-87e945ac-5a23-4ec3-a9b5-7df870375665 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.173271 May 23 16:14:36.437228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1747/3387] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:36 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:36.651742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:36.652273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:36.658960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.659446 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.660210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.660656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.797945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:36.812012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-addf7be7-8b28-4f5a-8613-70164b4539e4 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.162788 May 23 16:14:36.812012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1641/3388] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:36 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:36.881775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:36.882417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:36.889197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.889644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.890397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.890831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.910644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:36.919626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:36.921703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:36.926536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.926990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.927843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:36.928279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:36.955919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:36.964277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:36.966311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:36.967491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-26b65d4c-65b8-41f1-9ee2-478dad8e11eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089640 May 23 16:14:36.968055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1748/3389] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:36 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:37.006662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:37.014677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-33327850-e142-4359-b4c9-341a218b4758 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.100090 May 23 16:14:37.015269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1642/3390] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:36 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:37.453573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:37.454202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:37.460309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.460764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:37.461525 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.461963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:37.589729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:37.598548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-515e1d4d-0875-497a-a182-5650b4513a29 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149330 May 23 16:14:37.599189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1749/3391] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:37 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:37.829766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:37.830302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:37.836358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.836816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:37.837573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.838009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:37.969538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:37.977427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-361a69a4-2bed-4f89-8a20-163c48eaad9f tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.151590 May 23 16:14:37.978041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1643/3392] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:37 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:37.985132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:37.985741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:37.992903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.993562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:37.994625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:37.995355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:38.032139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:38.032793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:38.039058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:38.039546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:38.040628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:38.041066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:38.046198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:38.063116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:38.071732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:38.073255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:38.074910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7fa1e1ed-26dd-4ea7-b8c8-093fe57f581e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.093700 May 23 16:14:38.075812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1750/3393] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:37 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:38.104165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:38.112219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-46cde3fc-5215-4fdf-8242-3e125b4ab6d2 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084882 May 23 16:14:38.112790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1644/3394] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:38 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:38.617320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:38.617932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:38.623954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:38.624400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:38.625144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:38.625577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:38.749675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:38.757495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5340c0fb-e6d3-4b4a-98cc-100015f2b7d4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144122 May 23 16:14:38.758065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1751/3395] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:38 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:38.994755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:38.995403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:39.002134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.002583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.003364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.003797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.093539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:39.094175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:39.100654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.101095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.101845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.102274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.134501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.143047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-56317e52-cc97-4ec5-9eb4-7fd24db3c131 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.152051 May 23 16:14:39.143654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1645/3396] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:38 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:39.148477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:39.149094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:39.155532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.155982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.156733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.157162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.164103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.172543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.174141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.175405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e7a5d3b6-9008-4ab9-a15b-e93d00577168 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086218 May 23 16:14:39.176055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1752/3397] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:39 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:39.217490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.225411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-563010a9-f70c-42ec-90e8-e259bfb8dad4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.080827 May 23 16:14:39.225972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1646/3398] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:39 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:39.774665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:39.775454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:39.782209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.782669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.783501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:39.783895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:39.913324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:39.921213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-07a72223-de65-4630-aef6-9a9a72718e19 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.150382 May 23 16:14:39.921779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1753/3399] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:39 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:40.161158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:40.161711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:40.168245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.168700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.169462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.169900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.193654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:40.194270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:40.201301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.201754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.202507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.202941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.268447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:40.278338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:40.279888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:40.281051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4c2c80cd-ceb4-4cb7-b00e-db74d9056f89 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.091316 May 23 16:14:40.281622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1754/3400] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:40 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:40.286284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:40.286903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:40.293613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.294076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.294935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.295404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.326355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:40.334321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c242d80-474d-43cc-a9d5-4e5a1768b562 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.177125 May 23 16:14:40.334897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1647/3401] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:40 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:40.359682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:40.367578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1738d38e-f0e8-4014-a050-96cccbf44ddc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.084982 May 23 16:14:40.368144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1755/3402] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:40 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:40.938268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:40.938912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:40.944871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.945314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:40.946064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:40.946500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.070167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.078860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1347c55e-452a-4537-a920-50a199dda458 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144494 May 23 16:14:41.079457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1648/3403] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:40 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:41.299727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:41.300285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:41.306710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.307189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.307937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.308371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.351372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:41.351984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:41.358873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.359357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.360114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.369275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.378416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.379944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.381090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-effb8e79-a2f3-44a0-8ea8-9cbc5fac1188 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085319 May 23 16:14:41.381652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1756/3404] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:41 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:41.395403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:41.396022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:41.402100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.403086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.403879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:41.404318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:41.484130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.492340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-663c3bad-a7e4-4bf9-8bd2-8290f4694317 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.107864 May 23 16:14:41.492932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1757/3405] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:41 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:41.515524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:41.520939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f1f0b749-0c22-47db-8f81-ea1fb9c6aecd tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173428 May 23 16:14:41.521512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1649/3406] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:41 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:42.096333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:42.096995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:42.103183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.103640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.104397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.104837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.236789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.245152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-674c16c6-d53f-42d3-94cf-282ab48888c4 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152633 May 23 16:14:42.245706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1758/3407] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:42 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:42.399483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:42.400011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:42.406970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.407453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.408209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.408644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.468376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.476595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.478087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.479254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0d258a31-1506-4706-bbab-5c3865f2f95e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.083741 May 23 16:14:42.479808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1650/3408] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:42 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:42.509184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:42.509803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:42.516986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.517438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.518235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.518700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.537487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:42.538131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:42.544588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.545046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.545800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:42.546238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:42.591228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.599378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1666d2fb-24b2-4926-94e7-d47ff7f274ad tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.093917 May 23 16:14:42.599955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1759/3409] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:42 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:42.678570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:42.687051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f45a1a5-54b5-4e98-b1cf-f499f51fc23c tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.153327 May 23 16:14:42.687641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1651/3410] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:42 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:43.262247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:43.262864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:43.268914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.269361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.270371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.270805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.397937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.406452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cea8da41-da50-443e-aa9c-149918339525 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148596 May 23 16:14:43.407011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1760/3411] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:43 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:43.498438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:43.499150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:43.507909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.508364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.509133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.509573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.571274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.580375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.583329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.583329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-28729c4e-0e5d-4b46-bf9b-99460d25099d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.089271 May 23 16:14:43.583706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1652/3412] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:43 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:43.616773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:43.617393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:43.624304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.624304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.625008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.625452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.687319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.695296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0933096d-a40b-4c98-852a-02143d03e7b8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082299 May 23 16:14:43.695867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1761/3413] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:43 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:43.704666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:43.705284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:43.711394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.711847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.712605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:43.713041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:43.838889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:43.847336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5d9a7ab4-fc86-445b-b3ec-c3ef316d0a59 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146516 May 23 16:14:43.847903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1653/3414] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:43 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:44.423960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:44.424576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:44.430818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.431305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.432063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.432499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.574183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:44.584276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c4532334-4756-4eba-b7b6-b7c373569dbd tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.164760 May 23 16:14:44.585040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1762/3415] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:44 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:44.601541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:44.602179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:44.609882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.610324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.611074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.611527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.676244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:44.688701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:44.690311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:44.691389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4f8da7f7-4216-42c4-b927-a11bef14192f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.093976 May 23 16:14:44.691946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1654/3416] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:44 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:44.712645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:44.713316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:44.720367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.720859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.721618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.722057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.787786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:44.795768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-358a4f8d-147b-43c0-963d-7d367a8df27f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087186 May 23 16:14:44.796335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1763/3417] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:44 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:44.865011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:44.865620 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:44.871636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.872137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:44.872887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:44.873324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.012012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.022280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe3576e5-ae24-467c-9b44-3b286fe9f5e6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.160970 May 23 16:14:45.022861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1655/3418] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:44 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:45.604329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:45.604953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:45.611014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.611500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.612266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.612704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.709449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:45.711353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:45.717473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.717473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.718188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.718618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.741754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.749925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74220089-a995-4537-a757-31e4f7b03468 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149423 May 23 16:14:45.756692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1764/3419] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:45 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:45.782789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.791424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.792942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.794077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f3215b2d-3989-430c-8651-1e9a1a87a25d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088742 May 23 16:14:45.794628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1656/3420] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:45 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:45.812426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:45.813038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:45.819009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.819560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.820304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:45.820732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:45.825060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:45.883655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:45.892009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc40a74c-4127-4796-b447-d67b77394c43 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083375 May 23 16:14:45.892578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1765/3421] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:45 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:46.039552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:46.040003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:46.046060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.048881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.048881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.048881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.174341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.182255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a027ec9c-ba4f-4f74-b1e7-c8648dad9d6d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146819 May 23 16:14:46.182827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1657/3422] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:46 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:46.766454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:46.767060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:46.774326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.774782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.775566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.775999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.812030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:46.812766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:46.819382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.819836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.820603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.821042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.893941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.903122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.904775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.905923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ac8982a0-25a9-473e-9f3f-87709dd840b2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.097903 May 23 16:14:46.906485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1658/3423] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:46 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:46.910894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:46.911531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:46.918452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.918907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.919703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:46.920142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:46.924832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.933106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-35cf8d60-3f3b-4e45-87b2-cd6c3a997543 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.170496 May 23 16:14:46.933770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1766/3424] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:46 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:46.979344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:46.987361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6dc795f4-a756-41b9-b6d3-4ea9baad9ffe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079885 May 23 16:14:46.987932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1659/3425] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:46 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:47.201262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:47.201807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:47.207822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.208266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.209016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.209449 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.336138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:47.344176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-04d3960d-96c6-46a2-8f5c-f708150874f6 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146892 May 23 16:14:47.344753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1767/3426] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:47 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:47.924227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:47.924843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:47.931331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.931782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.932532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.932965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.950105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:47.951241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:47.957267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.957717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.958543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:47.958981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:47.999412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.002830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:48.008286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.009782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.010929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0575fd50-c7aa-4014-a9e0-879632897191 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.090570 May 23 16:14:48.011529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1660/3427] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:47 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:48.015845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:48.024216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:48.024216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:48.024216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:48.024216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:48.024659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:48.094923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.102699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-22e06a6d-f57b-4d69-8269-7da1e2abe44b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.090231 May 23 16:14:48.103292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1661/3428] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:48 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:48.118920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.127789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0dded9e9-1315-4ca2-8bfd-1e5796fdde43 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.181570 May 23 16:14:48.128361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1768/3429] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:47 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:48.362313 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:48.362937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:48.369370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:48.369846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:48.370645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:48.371114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:48.507589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:48.516018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5195ad8a-5c1e-4a72-8c89-96e8b0b25a80 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.157706 May 23 16:14:48.516602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1662/3430] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:48 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:49.028906 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:49.029511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:49.035993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.036439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.037188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.037621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.101795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.110156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.111691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.112841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-60008b0c-38d9-441e-a132-1b3615936da4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087819 May 23 16:14:49.113398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1769/3431] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:49 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:49.120200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:49.121421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:49.128118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.128571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.129348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.129768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.144084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:49.144958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:49.150922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.151399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.152157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.152585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.194570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.204434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-786f5e34-a95b-4e45-8783-d3323b263c67 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.088172 May 23 16:14:49.205090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1663/3432] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:49 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:49.289778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.297694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c80c5376-d754-4041-bdbe-fa3a12c5b3c8 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.157569 May 23 16:14:49.298262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1770/3433] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:49 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:49.536205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:49.539563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:49.543361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.543805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.544552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:49.544985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:49.672706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:49.680739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e5c1c125-0388-4b8b-9585-d9e85cd5791a tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.148965 May 23 16:14:49.681303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1664/3434] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:49 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:50.131215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:50.131856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:50.138832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.139318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.140079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.140512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.202013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.210625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.212169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.213311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ec975e95-b5ea-4944-9fbf-3edb75593289 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086072 May 23 16:14:50.213878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1771/3435] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:50 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:50.223086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:50.224219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:50.233636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.234308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.235586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.236244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.304960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.312831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-066ce979-c3b3-4c88-8cb2-e1804a1fa9f4 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.095097 May 23 16:14:50.313389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1665/3436] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:50 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:50.313981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:50.314694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:50.320689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.321142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.321897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.322335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.453398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.461255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a9576f00-303a-483f-9901-06691c84b5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.151241 May 23 16:14:50.461813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1772/3437] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:50 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:50.698333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:50.699592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:50.705550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.705993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.706739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:50.707203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:50.832870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:50.841293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a99bc6c9-87a9-42b4-ba4f-0beba7db7356 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146917 May 23 16:14:50.841844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1666/3438] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:50 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:51.233458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:51.234609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:51.244707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.245477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.246775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.247573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.314421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:51.322770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:51.324305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:51.325450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4651fa54-740c-4860-8ef9-f2d24445f9ab tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.097339 May 23 16:14:51.326007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1773/3439] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:51 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:51.328730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:51.329380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:51.335970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.336426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.337193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.337633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.398170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:51.406426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3af90e58-8271-4d4d-9818-61d3c5245f13 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081729 May 23 16:14:51.406980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1667/3440] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:51.477635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:51.478248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:51.484892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.485286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.486035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.486469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.616687 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:51.624648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ccc5719b-bd43-495f-9f15-7b20b60ba69b tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.150864 May 23 16:14:51.625228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1774/3441] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:51 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:51.858376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:51.858913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:51.864968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.865419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:51.866178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:51.866611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.001067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.008987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-782bccb8-5cf5-4a03-9a44-ef57d22c0137 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.155056 May 23 16:14:52.009547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1668/3442] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:51 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:52.343726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:52.344339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:52.350807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.351296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.352055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.352495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.412763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.421189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.422742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.423060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:52.423698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:52.423916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-72f0c751-fefc-4267-ac26-a2cc45c46869 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084814 May 23 16:14:52.424654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1775/3443] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:52 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:52.430059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.430505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.431290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.431720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.497120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.504850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e1b9f95e-3841-4d29-9fc2-d38d403d2470 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.085600 May 23 16:14:52.505415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1669/3444] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:52 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:52.642030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:52.642621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:52.648779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.649221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.649966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:52.650397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:52.775935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:52.784178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dae3fb8b-56e0-42f0-afeb-9c439ee88109 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.145921 May 23 16:14:52.784756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1776/3445] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:52 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:53.025963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:53.026512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:53.033448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.033883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.034642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.035145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.161100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.169030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f83818e7-af32-4f0a-ad73-dfe154f46c0e tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.146960 May 23 16:14:53.169605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1670/3446] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:53 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:53.443773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:53.444324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:53.446657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:53.451863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.452320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.453081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.453524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.514376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.521909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:53.522518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:53.523024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.524564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.524896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:53.525712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e1da9d8d-1b04-45da-ae03-a4b8955ff208 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.085982 May 23 16:14:53.526588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1777/3447] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:53 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:53.530189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.530640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.531495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.531953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.592840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.600697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-87b1dde8-e237-4136-8e6c-8c2c5355789a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082837 May 23 16:14:53.601267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1671/3448] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:53 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:53.802178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:53.802800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:53.809064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.809514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.810303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:53.810746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:53.937692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:53.945548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e22e25ad-a506-41e5-a8e2-5d3ab759f040 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147212 May 23 16:14:53.946116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1778/3449] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:53 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:54.192893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:54.193426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:54.199986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.200432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.201178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.201607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.326592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:54.334523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61da3073-0abc-4775-a9bf-03f00f3c465d tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145527 May 23 16:14:54.335077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1672/3450] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:54 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:54.544215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:54.544837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:54.551461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.551912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.552673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.553114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.616678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:54.617711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:54.618339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:54.625448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.625747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:54.625898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.626657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.627124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.627256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:54.628403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3f50e1c9-68c6-4bd6-8208-5c9debeec940 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.088618 May 23 16:14:54.628960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1779/3451] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:54 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:54.688852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:54.696702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ee8ecd6-0d04-4925-bb51-dc31fe02771b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.082784 May 23 16:14:54.697266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1673/3452] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:54 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:54.962657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:54.963261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:54.969757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.970200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:54.970950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:54.971413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.097199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.105177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-39339db9-ada5-4297-95c0-c48fd89bd054 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.146435 May 23 16:14:55.106266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1780/3453] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:54 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:55.352474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:55.353118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:55.359491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.359950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.360716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.361161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.489432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.497967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0787bca9-45cf-4313-9aa4-072453912a40 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149646 May 23 16:14:55.498527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1674/3454] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:55 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:55.646603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:55.647339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:55.654310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.654833 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.655674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.656108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.713496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:55.714246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:55.718040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.720871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.721322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.721456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:55.722086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:55.722538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:55.727069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.728621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.729834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-623acb35-e002-40b0-bee5-c46ae18b8748 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087376 May 23 16:14:55.731071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1781/3455] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:55 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:55.790490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:55.799317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9cae00e-e749-428e-89f5-ac74f513bfa9 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.089667 May 23 16:14:55.799922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1675/3456] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:55 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:56.122891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:56.123447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:56.129782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.130226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.130973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.131429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.258350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.266794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-871acf5f-8c6c-4cdf-b012-58a6b54bfeec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.148209 May 23 16:14:56.267398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1782/3457] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:56 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:56.515574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:56.516182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:56.522793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.523271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.524023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.524454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.648619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.656525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62d414db-fe2a-4cad-8a16-1dd89deb1319 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.145511 May 23 16:14:56.657089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1676/3458] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:56 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:56.748427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:56.749039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:56.755410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.755855 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.756600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.757044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.816670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:56.817290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:56.817467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.823305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.823769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.824558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:56.824997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:56.825828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.827404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.828532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b5b9436d-7e9f-4d69-b23b-79bf0dc4fbfd tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.084638 May 23 16:14:56.829083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1783/3459] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:56 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:14:56.842962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:56.887923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:56.895914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35d6a6fa-9032-4c2f-a8b1-e20c2c8a2b88 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.083080 May 23 16:14:56.896492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1677/3460] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:56 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:57.284559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:57.285173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:57.291173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.291626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.292386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.292822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.420361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.428154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a70b6623-4286-4b77-ba4e-063e3708e5ec tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147456 May 23 16:14:57.428715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1784/3461] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:57 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:57.673422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:57.673954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:57.680421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.680875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.681629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.682065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.804901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.812755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-962d0131-dbe1-4dbd-9b8a-4d5d0d73c0ff tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.143280 May 23 16:14:57.813318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1678/3462] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:57 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:57.846698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:57.847346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:57.853791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.854242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.854993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.855455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.912371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:57.913026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:57.918972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.919223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.919609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.920366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:57.920799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:57.927835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.929429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.930577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e87d5ba-a178-4a3f-86c6-b4e978d35410 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.087706 May 23 16:14:57.931162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1785/3463] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:57.980305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:57.988114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-182cd145-51c3-4e67-9c71-22ff7ee0bc4d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.079658 May 23 16:14:57.988723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1679/3464] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:57 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:58.010827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:14:58.011004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:14:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:14:58.445042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:58.445664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:58.453945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.454402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.456435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.456892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.585120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:58.593445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83775a6a-3e74-46e8-85fb-929c7dc769a1 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.152279 May 23 16:14:58.594011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1786/3465] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:58 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:58.829029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:58.829609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:58.836061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.836505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.837250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.837681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.949195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:58.949811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:58.956275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.959562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.959562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:58.959562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:58.969957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:58.978292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a79ea4e3-2255-458c-b585-8aab43172a58 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.153173 May 23 16:14:58.978862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1680/3466] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:58 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:59.005306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:59.005919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:59.011912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:59.012359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:59.013609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:59.014041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:59.020220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:59.028986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:59.030488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:59.031665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4eb504df-b615-4f3f-b094-2ef65783b20b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086367 May 23 16:14:59.032230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1787/3467] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:14:58 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:14:59.074447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:59.082893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ff991f49-b474-4f85-bd81-a48cef6300dc tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.081435 May 23 16:14:59.083436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1681/3468] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:59 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:14:59.610520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:59.611222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:14:59.617243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:59.617691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:59.618442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:14:59.618875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:14:59.745812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:14:59.754241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af98837c-34a2-4ba4-a8b9-ff354732b87e tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.147607 May 23 16:14:59.754798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1788/3469] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:59 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:14:59.996055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:14:59.997929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:00.004991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.005445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.006213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.006680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.049986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:00.050583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:00.057003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.057455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.058216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.058655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.136812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.145422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.146923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.148098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3daade49-2af0-47f8-9311-de3976bc7b43 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.101988 May 23 16:15:00.148661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1789/3470] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:00 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:15:00.157810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.160308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:00.161040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:00.166019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a4aba70-9e81-486c-8beb-2d3f4e2bb9e3 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.173863 May 23 16:15:00.166671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1682/3471] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:14:59 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:00.166937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.167422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.168198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.168640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.228602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.237325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ef80797c-7437-444d-b47b-12b74f366579 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087683 May 23 16:15:00.237941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1790/3472] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:00 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:00.770825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:00.771499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:00.777485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.777940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.779328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:00.779770 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:00.908401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:00.916269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-65cef64b-fae3-412e-baa5-234f8870453c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.149521 May 23 16:15:00.916852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1683/3473] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:00 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:01.166410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:01.166966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:01.173775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.174231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.174992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.175464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.183950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:01.185066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:01.191498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.191949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.192710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.193149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.245788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:01.255006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:01.256539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:01.257689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-050d3f48-7098-4124-9a64-4b984e5dc07b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.095246 May 23 16:15:01.258234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1791/3474] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:01 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:01.262516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:01.263142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:01.269627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.270475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.271338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.271781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.342683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:01.350496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8d2309c8-f75f-4431-a41c-544251c331af tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091321 May 23 16:15:01.351073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1792/3475] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:01 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:01.359027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:01.366913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84d79d4d-91c6-4069-b6a5-c0e0950fe119 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.186864 May 23 16:15:01.367494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1684/3476] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:01 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:01.933299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:01.933841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:01.939982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.940435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:01.941195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:01.941637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.068703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.074102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f163adef-ed7c-44fd-8294-a850f274ce83 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144776 May 23 16:15:02.074670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1793/3477] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:01 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:02.275570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:02.276185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:02.283414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.283815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.284571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.285007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.346792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.355226 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.356739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.357871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7490b02e-df8e-422d-8117-953adcdbf134 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.086150 May 23 16:15:02.358426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1685/3478] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:02 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:02.367502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:02.368116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:02.374936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.375419 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.376177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.376614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.384151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:02.384793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:02.391258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.391715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.392478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:02.392914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:02.442798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.451480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ba458034-6926-48ad-8e95-7409020197e5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.087643 May 23 16:15:02.452060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1794/3479] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:02 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:02.538928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:02.546980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b0f18f81-6dc9-4aae-a10c-2a02e666c9cf tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.166507 May 23 16:15:02.547571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1686/3480] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:02 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:03.092447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.093054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] [instance: 087911cb-6647-4769-a496-f0a0f2e0482c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:03.099030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.099518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.100278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.100716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.224034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '16a2ff4a80da41bf84f19450000a9da1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ce8b36a28714d618becacd8e2925a75', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.232011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76d820b6-24dc-4817-a3cf-4f7d6eff9f0c tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c" status: 200 len: 1601 microversion: 2.1 time: 0.144179 May 23 16:15:03.232588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1795/3481] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:03 2022] GET /compute/v2.1/servers/087911cb-6647-4769-a496-f0a0f2e0482c => generated 1601 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:03.248727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cd23e5b0-5892-4907-af46-0dd0c8bed0a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] Calling method '.version_select at 0x7f47f9bb0dc0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.345643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cd23e5b0-5892-4907-af46-0dd0c8bed0a2 tempest-TestMinimumBasicScenario-2097968565 tempest-TestMinimumBasicScenario-2097968565-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1208197562" status: 202 len: 0 microversion: 2.1 time: 0.101178 May 23 16:15:03.346260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1687/3482] 10.222.0.32 () {58 vars in 1271 bytes} [Mon May 23 16:15:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1208197562 => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:15:03.375454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.376062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:03.382121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.382721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.383495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.383929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.448414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.463032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.465122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.466590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e91ecd4c-efe9-4881-8dbb-9d0778a0b2fe tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 200 len: 1444 microversion: 2.20 time: 0.095700 May 23 16:15:03.467558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1796/3483] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:03 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 1444 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:03.469155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.470017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:03.476554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.477007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.477974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.478587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.487354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a93e8a01-caca-41b5-a5bb-c111974b2d3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '.version_select at 0x7f47f927cc10>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.549173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.555623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6468fd9c-9859-4e93-852a-9fdbe0c2e4fb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 200 len: 1373 microversion: 2.1 time: 0.091809 May 23 16:15:03.556198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1688/3484] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:03 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:03.564328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.565039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] [instance: 3a14c169-ee02-4abd-8bae-be450c73183c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:03.571003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.571499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.572262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:03.572700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:03.643385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a93e8a01-caca-41b5-a5bb-c111974b2d3c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1200855778" status: 204 len: 0 microversion: 2.20 time: 0.160419 May 23 16:15:03.644533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1797/3485] 10.222.0.32 () {60 vars in 1261 bytes} [Mon May 23 16:15:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1200855778 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 23 16:15:03.649787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1a9126e-8482-44d0-b7c0-8ea49fa70a41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '.version_select at 0x7f47f9072ca0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:03.702089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cc4c3405e93141fc851f22e49eee09bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e22027c2e5948a6ba154c837a7673a9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:03.710172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-52f552ed-523c-4b6e-a54e-c5491dec6f90 tempest-VolumesSnapshotTestJSON-577538905 tempest-VolumesSnapshotTestJSON-577538905-project] 10.222.0.32 "GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c" status: 200 len: 1558 microversion: 2.1 time: 0.149555 May 23 16:15:03.710735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1689/3486] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:03 2022] GET /compute/v2.1/servers/3a14c169-ee02-4abd-8bae-be450c73183c => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:03.745721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1a9126e-8482-44d0-b7c0-8ea49fa70a41 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1265677059" status: 202 len: 0 microversion: 2.1 time: 0.100624 May 23 16:15:03.746539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1798/3487] 10.222.0.32 () {58 vars in 1220 bytes} [Mon May 23 16:15:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1265677059 => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:15:04.662931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b40d918e-56ee-4480-9998-936d5decaf35 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: .version_select at 0x7f47f9b47c10>, body: {"keypair": {"name": "tempest-keypair-113322561", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCLy3oCoe2Hv64yQoB1dIT9rqRxSZRdnjUT6EIF2xHiMa+HezBMC1jM2rXHdIp1LvRETPkwcDWtSfSkWlXIYBJ3kIOhycLIyy6jKgdOrw8X8KLV1RDDfKZMGJWywVokBmA=="}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:04.679743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-b40d918e-56ee-4480-9998-936d5decaf35 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:15:04.683660 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-b40d918e-56ee-4480-9998-936d5decaf35 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for user 6d689e7ca175463ab0c319260ba49d0d and project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:15:04.724384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c43291a1-3296-46a6-b900-0cc004191c24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: .version_select at 0x7f47feb03670>, body: {"keypair": {"name": "tempest-keypair-787462989", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG1K+AMdIbwzOFEGK8IOqIEY4qJNVAcp2udcfP+vjFgfYmGvmyGtjEWjDzHaq4HqxaPpBkAScCVJqGMLQqBijsGlKp4Q4Cf1N5vc9zE31DTjT0MuWrl9bewSzwuW9d3lqw=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:04.739607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-c43291a1-3296-46a6-b900-0cc004191c24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:15:04.744808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-c43291a1-3296-46a6-b900-0cc004191c24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:15:04.764094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b40d918e-56ee-4480-9998-936d5decaf35 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.105062 May 23 16:15:04.765234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1690/3488] 10.222.0.32 () {62 vars in 1199 bytes} [Mon May 23 16:15:04 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 107 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) May 23 16:15:04.814812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c43291a1-3296-46a6-b900-0cc004191c24 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.094817 May 23 16:15:04.815601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1799/3489] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 16:15:04 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:15:08.013806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1443478562", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7e7a8587-a97e-4c11-9c4a-682adeb9c236"}], "security_groups": [{"name": "tempest-securitygroup--1468827874"}], "key_name": "tempest-keypair-787462989", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:08.181348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] validate_networks() for [('7e7a8587-a97e-4c11-9c4a-682adeb9c236', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 16:15:08.331317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1793356317", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5be72b67-de15-4ced-95f8-5cd9e93f3a7a"}], "security_groups": [{"name": "tempest-securitygroup--47953194"}], "key_name": "tempest-keypair-113322561", "user_data": "}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:08.503633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] validate_networks() for [('5be72b67-de15-4ced-95f8-5cd9e93f3a7a', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 16:15:08.886252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 16:15:08.886828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 16:15:08.887384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 16:15:08.887775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 16:15:08.888294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 16:15:08.897529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:08.897724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:08.898517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:08.898953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:08.915625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:15:08.920102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:15:08.931062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 16:15:08.933401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 16:15:09.085288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fb37bfa9-6e2a-4d9a-b1e6-c65f40a392c8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.076836 May 23 16:15:09.085957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1691/3490] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 16:15:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 1078 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 16:15:09.092073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 16:15:09.092546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 16:15:09.093028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 16:15:09.093594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 16:15:09.093840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 16:15:09.096535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7860c76b-0b37-468d-ae4b-28a0c04f11d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:09.097523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7860c76b-0b37-468d-ae4b-28a0c04f11d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:09.105485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:09.107373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:09.107717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:09.108360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:09.128358 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:15:09.132352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Getting quotas for user 6d689e7ca175463ab0c319260ba49d0d and project 7651f704e84b4ce8bbd53a6292126bce. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:15:09.144645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 16:15:09.147432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 16:15:09.155252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7860c76b-0b37-468d-ae4b-28a0c04f11d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:09.163456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7860c76b-0b37-468d-ae4b-28a0c04f11d3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1241 microversion: 2.1 time: 0.071016 May 23 16:15:09.171527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1692/3491] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:09 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1241 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:09.258893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR oslo.messaging._drivers.impl_rabbit [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [59956294-577a-4c52-abed-40fd2c04fc0e] AMQP server on 10.222.0.32:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 16:15:10.180925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:10.181550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:10.188545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:10.188994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:10.189757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:10.190193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:10.251493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:10.258927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2dd0607d-12cb-4a71-aa59-97afb57e3721 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1245 microversion: 2.1 time: 0.081831 May 23 16:15:10.259697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1693/3492] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:10 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1245 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:10.268947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [59956294-577a-4c52-abed-40fd2c04fc0e] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 55402. May 23 16:15:10.273541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9bc9ef9d-e408-47f0-bed1-f2f041ac5f5a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 1.947174 May 23 16:15:10.274262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1800/3493] 10.222.0.32 () {62 vars in 1187 bytes} [Mon May 23 16:15:08 2022] POST /compute/v2.1/servers => generated 400 bytes in 1949 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) May 23 16:15:10.284142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:10.288847 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:10.342945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:10.350112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:10.351657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:10.352772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5383e8eb-dda6-4d88-b665-48c9bca55406 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1281 microversion: 2.20 time: 0.072579 May 23 16:15:10.353429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1694/3494] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:10 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1281 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:11.276648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:11.277250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:11.284121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:11.284564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:11.285309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:11.285737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:11.352923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:11.373360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7ffac2cf-04b6-4029-b672-a6357e587df5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1311 microversion: 2.1 time: 0.100847 May 23 16:15:11.382920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1801/3495] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:11 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1311 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:11.384331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:11.384331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:11.432652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:11.440944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:11.442470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:11.443606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b63ce2c8-52f3-4477-849d-d4e1772dbe71 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1281 microversion: 2.20 time: 0.075946 May 23 16:15:11.444167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1695/3496] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:11 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1281 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:12.389992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:12.391682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:12.399868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:12.400322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:12.401084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:12.401518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:12.406162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:12.461628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:12.462435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:12.529697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:12.538676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:12.539070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:12.540282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-23eaadbd-886e-426a-b274-c96577801de3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1281 microversion: 2.20 time: 0.082615 May 23 16:15:12.543163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1696/3497] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:12 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1281 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:15:12.581744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:12.590139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eb15b49a-91af-46c3-bc2e-9e7c44256b40 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.204193 May 23 16:15:12.601174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1802/3498] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:12 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:12.968304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd", "tag": "243e14c9-e626-4c45-8053-04ff3f1602dd"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:12.977256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:12.977698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:12.988602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] Creating event network-changed:243e14c9-e626-4c45-8053-04ff3f1602dd for instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd on hv-cinder-83588 May 23 16:15:12.992982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR oslo.messaging._drivers.impl_rabbit [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] [25d4d26c-ffcb-418e-a94d-5772f6611074] AMQP server on 10.222.0.32:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 16:15:13.557775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:13.558393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:13.614755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:13.621937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:13.623471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:13.624573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ed57790c-511f-45b2-863d-1b77113637dc tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1281 microversion: 2.20 time: 0.070791 May 23 16:15:13.625133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1803/3499] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:13 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1281 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:15:13.629413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:13.629999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:13.636194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:13.636671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:13.637425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:13.637882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:13.768864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:13.776621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d996f5c-9ef5-4a51-b5ed-6a4ce64e5f23 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.150559 May 23 16:15:13.777182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1804/3500] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:13 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:13.999745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] [25d4d26c-ffcb-418e-a94d-5772f6611074] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 55440. May 23 16:15:14.003515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-65d3b95a-55cf-46da-921d-833fe6cde127 req-472a65f7-0e3f-4bac-a172-9cc8fac21301 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 1.105646 May 23 16:15:14.004114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1697/3501] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 1107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:14.643374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:14.643994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:14.650704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:14.651185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:14.651939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:14.652377 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:14.712805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:14.720628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:14.722129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:14.723246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-547510df-4c47-4eea-a615-3bbe4ae2a332 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1285 microversion: 2.20 time: 0.083838 May 23 16:15:14.723797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1805/3502] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:14 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1285 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:14.795312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:14.796015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:14.802162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:14.802633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:14.803389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:14.803829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:14.932563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:14.940429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6c41f6c5-4849-4e21-b371-9ba351d63ec8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.150306 May 23 16:15:14.940994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1698/3503] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:14 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:15.741592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:15.742355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:15.748913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:15.749367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:15.750127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:15.750571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:15.811811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:15.819773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:15.821276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:15.822380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c536a47-91a3-4a84-a048-f1ef86694579 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1285 microversion: 2.20 time: 0.084715 May 23 16:15:15.822933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1806/3504] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:15 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1285 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:15.957056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:15.957702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:15.964151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:15.964631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:15.965390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:15.965835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:16.100299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:16.108233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-205b96b8-9e32-4853-8bc9-5712722cd283 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.154853 May 23 16:15:16.108810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1699/3505] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:15 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:16.841233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:16.841846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:16.848354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:16.848799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:16.849553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:16.849986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:16.915310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:16.924084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:16.925913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:16.927023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-20783d62-a6c6-4696-96cc-3482f0c308c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1333 microversion: 2.20 time: 0.090306 May 23 16:15:16.927626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1807/3506] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:16 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1333 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:17.124802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:17.125359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:17.131901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:17.132349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:17.133099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:17.133533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:17.264802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:17.272449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b23fe10b-4a00-4fae-866e-496bb4890ed5 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.151532 May 23 16:15:17.273032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1700/3507] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:17 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:17.945437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:17.946060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:17.952645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:17.953100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:17.953874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:17.954319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:18.091802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:18.100328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:18.101896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:18.103369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-63b22fe6-db4f-4def-9e4f-5bf1e646cbe6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.162020 May 23 16:15:18.103926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1808/3508] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:17 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:18.291713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:18.292431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:18.302438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:18.302837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:18.303772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:18.304211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:18.323911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:18.446572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:18.455870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7f668e5b-ae60-4a58-8868-3200ba985901 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.168535 May 23 16:15:18.456648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1701/3509] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:18 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:18.797083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:15:18.797347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:15:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:15:19.015621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-96deb7d7-b731-4d8f-8234-ebc995600c6b req-9e0ab576-435f-4dff-9f8b-bbf44b609fc3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:19.026970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-96deb7d7-b731-4d8f-8234-ebc995600c6b req-9e0ab576-435f-4dff-9f8b-bbf44b609fc3 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:19.027489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-96deb7d7-b731-4d8f-8234-ebc995600c6b req-9e0ab576-435f-4dff-9f8b-bbf44b609fc3 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:19.039596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-96deb7d7-b731-4d8f-8234-ebc995600c6b req-9e0ab576-435f-4dff-9f8b-bbf44b609fc3 service nova] Creating event network-changed:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:15:19.045955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-96deb7d7-b731-4d8f-8234-ebc995600c6b req-9e0ab576-435f-4dff-9f8b-bbf44b609fc3 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036915 May 23 16:15:19.049702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1809/3510] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:19.123649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:19.125417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:19.132925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:19.134995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:19.135543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:19.136332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:19.136797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:19.295557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:19.304854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:19.306375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:19.307833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8e72f171-0cee-4e0c-a7c4-ba6e34134a74 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.189909 May 23 16:15:19.308603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1702/3511] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:19 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:19.470034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:19.474249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:19.484018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:19.484018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:19.484018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:19.484018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:19.484583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:19.617351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:19.625125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2dda5ffe-585e-4b49-921c-fac90c234d17 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.155997 May 23 16:15:19.625636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1810/3512] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:19 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:20.326371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:20.326986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:20.329327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:20.334520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:20.334993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:20.339772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:20.339772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:20.464433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:20.472336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:20.473838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:20.474941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ee89ae4d-7ea1-4595-8d4e-c58a89ef6896 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.153081 May 23 16:15:20.475535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1703/3513] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:20 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:15:20.642470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:20.643159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:20.645688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:20.650475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:20.650928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:20.651710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:20.652146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:20.768050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-ad89f515-25c7-463d-8e9a-4d1261a175ac req-d0b37a89-e8b8-465e-81b9-1c78b248ea4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd", "name": "network-vif-plugged", "status": "completed", "tag": "243e14c9-e626-4c45-8053-04ff3f1602dd"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:20.776477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-ad89f515-25c7-463d-8e9a-4d1261a175ac req-d0b37a89-e8b8-465e-81b9-1c78b248ea4d service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:20.776963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-ad89f515-25c7-463d-8e9a-4d1261a175ac req-d0b37a89-e8b8-465e-81b9-1c78b248ea4d service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:20.783702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:20.788300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-ad89f515-25c7-463d-8e9a-4d1261a175ac req-d0b37a89-e8b8-465e-81b9-1c78b248ea4d service nova] Creating event network-vif-plugged:243e14c9-e626-4c45-8053-04ff3f1602dd for instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd on hv-cinder-83588 May 23 16:15:20.791542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0039e364-c082-4527-a3fe-543237248800 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.153553 May 23 16:15:20.792499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1811/3514] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:20 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:20.803297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-ad89f515-25c7-463d-8e9a-4d1261a175ac req-d0b37a89-e8b8-465e-81b9-1c78b248ea4d service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097146 May 23 16:15:20.806843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1704/3515] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:15:21.493214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:21.494156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:21.504047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:21.504720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:21.505877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:21.506541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:21.638605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:21.646617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:21.648180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:21.649277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2400b74c-51c3-4179-8fb8-81f99fb3cd33 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.160460 May 23 16:15:21.649832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1812/3516] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:21 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:21.810012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:21.810644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:21.816874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:21.817329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:21.818090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:21.818530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:21.955840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:21.964103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1b69aebb-7719-4bd2-9b59-04dea657bf60 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.158851 May 23 16:15:21.964670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1705/3517] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:21 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:22.668196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:22.668822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:22.676180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:22.676638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:22.677395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:22.677831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:22.807046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:22.816230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:22.817747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:22.818914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ed71e91f-61bd-4681-85b6-c10bdad59ad9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.154803 May 23 16:15:22.819510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1813/3518] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:22 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:22.980807 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:22.981441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:22.983905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=18 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:22.989266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:22.989724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:22.990503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:22.990928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:23.122075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:23.129704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2bedc3f2-87d4-4ce5-ae48-0c178a69b893 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.152728 May 23 16:15:23.130194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1706/3519] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:22 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:23.842226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:23.842836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:23.844692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=12 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:23.849905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:23.850348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:23.851114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:23.851546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:23.980958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:23.989806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:23.991590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:23.994576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2efb9e85-56f3-41b7-8711-e3aec759ee53 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.154367 May 23 16:15:23.994576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1814/3520] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:23 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:24.146887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:24.147567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:24.154572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:24.155014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:24.155806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:24.156246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:24.282665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:24.290051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1b2fa2b9-1dbc-4384-8640-890ac7901a93 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1367 microversion: 2.1 time: 0.147228 May 23 16:15:24.290610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1707/3521] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:24 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1367 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:25.010979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:25.011777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:25.019506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.019952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.020961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.021391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.149119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:25.157060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:25.158607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:25.159761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-db2c9699-7104-4073-aba9-0538d6250615 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.152688 May 23 16:15:25.160316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1815/3522] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:25 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:25.307573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:25.308203 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:25.315032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.315518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.316279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.316716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.703846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:25.711303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a360e0f-59a4-4c8d-9dc0-a48a9c8e3494 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1557 microversion: 2.1 time: 0.407526 May 23 16:15:25.711874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1708/3523] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:25 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1557 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:25.727064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:25.727585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:25.733565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.734012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.734769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:25.735252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:25.809902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f4c8503c-8913-48e9-9439-27faedc6bc61 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.086062 May 23 16:15:25.810470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1816/3524] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 16:15:25 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-interface => generated 284 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:26.177657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:26.178338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:26.185200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:26.185656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:26.186420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:26.186853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:26.370712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:26.378980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:26.380548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:26.381668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2e83d7b7-20e2-4f63-9aab-cd7defacf8b1 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.207987 May 23 16:15:26.382285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1709/3525] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:26 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:26.772368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-4df5ba42-affa-41d9-a931-f3bdb9a7bcab req-f5af9584-bd32-4ab1-9244-62fd32c83ea7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "name": "network-vif-plugged", "status": "completed", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:26.781493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-4df5ba42-affa-41d9-a931-f3bdb9a7bcab req-f5af9584-bd32-4ab1-9244-62fd32c83ea7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:26.782082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-4df5ba42-affa-41d9-a931-f3bdb9a7bcab req-f5af9584-bd32-4ab1-9244-62fd32c83ea7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:26.792901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-4df5ba42-affa-41d9-a931-f3bdb9a7bcab req-f5af9584-bd32-4ab1-9244-62fd32c83ea7 service nova] Creating event network-vif-plugged:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:15:26.798202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-4df5ba42-affa-41d9-a931-f3bdb9a7bcab req-f5af9584-bd32-4ab1-9244-62fd32c83ea7 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030496 May 23 16:15:26.799077 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1817/3526] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:26.900494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-bdad042a-1e8e-4ac0-9f69-3bbac23079d0 req-e1913980-6f9d-40fa-89e2-37251b146eb8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd", "tag": "243e14c9-e626-4c45-8053-04ff3f1602dd"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:26.910854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-bdad042a-1e8e-4ac0-9f69-3bbac23079d0 req-e1913980-6f9d-40fa-89e2-37251b146eb8 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:26.911329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-bdad042a-1e8e-4ac0-9f69-3bbac23079d0 req-e1913980-6f9d-40fa-89e2-37251b146eb8 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:26.923240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-bdad042a-1e8e-4ac0-9f69-3bbac23079d0 req-e1913980-6f9d-40fa-89e2-37251b146eb8 service nova] Creating event network-changed:243e14c9-e626-4c45-8053-04ff3f1602dd for instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd on hv-cinder-83588 May 23 16:15:26.929275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-bdad042a-1e8e-4ac0-9f69-3bbac23079d0 req-e1913980-6f9d-40fa-89e2-37251b146eb8 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091551 May 23 16:15:26.931082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1710/3527] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:27.400890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:27.401492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:27.407600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:27.408049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:27.408799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:27.409233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:27.540332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:27.549289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:27.550804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:27.552001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6cf7e19-20d3-4d82-84c5-ab5bf17a4831 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.155485 May 23 16:15:27.552569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1818/3528] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:27 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:28.570927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:28.581100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:28.581100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:28.581100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:28.581100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:28.581541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:28.711360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:28.719530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:28.721045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:28.722152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c58db9e-bb84-4ee2-8772-f70168da59c3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1405 microversion: 2.20 time: 0.156050 May 23 16:15:28.722702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1711/3529] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:28 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1405 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:29.739355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:29.739973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:29.746499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:29.746948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:29.747734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:29.748165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:29.877436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:29.885363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:29.886849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:29.888552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45b2b72c-9770-4974-b3e1-8f31a61fb86c tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1597 microversion: 2.20 time: 0.153126 May 23 16:15:29.889113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1819/3530] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:15:29 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1597 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:15:29.907137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:29.909297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:29.914409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:29.914915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:29.915715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:29.916148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:29.990938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-730589b3-968b-47a4-8db2-b2c271827950 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.088806 May 23 16:15:29.991574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1712/3531] 10.222.0.32 () {60 vars in 1315 bytes} [Mon May 23 16:15:29 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-interface => generated 285 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:15:31.013121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-6ddebf5b-f83e-4962-bbcb-f15b71712c48 req-55d50845-7084-4b8a-943a-659a1d431dd9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:15:31.022037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-6ddebf5b-f83e-4962-bbcb-f15b71712c48 req-55d50845-7084-4b8a-943a-659a1d431dd9 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:31.022480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-6ddebf5b-f83e-4962-bbcb-f15b71712c48 req-55d50845-7084-4b8a-943a-659a1d431dd9 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:31.034569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-6ddebf5b-f83e-4962-bbcb-f15b71712c48 req-55d50845-7084-4b8a-943a-659a1d431dd9 service nova] Creating event network-changed:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:15:31.039819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-6ddebf5b-f83e-4962-bbcb-f15b71712c48 req-55d50845-7084-4b8a-943a-659a1d431dd9 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030236 May 23 16:15:31.040416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1820/3532] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:15:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:15:33.343576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:33.344120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:33.350590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.351085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.351911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.352358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.380323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:15:33.452220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:33.452799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:33.459005 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.461739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.461739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.461739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.485480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:15:33.500528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:33.650399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a99e416-65f0-413e-9a9d-478e2259840a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 204 len: 0 microversion: 2.1 time: 0.311294 May 23 16:15:33.650985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1713/3533] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:15:33 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:15:33.659500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:33.660499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:33.666297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.666742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.667576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.668011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.750486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2fa5e8f6-2890-4526-aaf4-620e29ae4a43 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 204 len: 0 microversion: 2.1 time: 0.303442 May 23 16:15:33.751083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1821/3534] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:15:33 2022] DELETE /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 23 16:15:33.759513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:33.760212 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:33.766284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.766732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.767515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:33.767948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:33.806648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:33.819319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-08b14c68-81cc-443b-8a51-37a30fe55f5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.161272 May 23 16:15:33.819947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1714/3535] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:33.904755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:33.912728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6d022c4-ec1f-4ab0-85fd-1092f5842138 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.157439 May 23 16:15:33.913291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1822/3536] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:34.836830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:34.837457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:34.844106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:34.844557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:34.845315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:34.845753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:34.930227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:34.930834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:34.937988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:34.938432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:34.939206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:34.939636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:34.976914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:34.985465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-50be21cc-1605-4543-acba-3b6bfc1e52f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.152490 May 23 16:15:34.986025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1715/3537] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:35.070336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:35.078651 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f50a9720-82c0-4155-a1b3-6ebbc4b8351c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.152109 May 23 16:15:35.079242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1823/3538] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:36.002362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:36.002973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:36.009767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:36.010220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:36.010991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:36.011466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:36.095804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:36.096425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:36.102597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:36.103050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:36.103845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:36.104289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:36.154286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:36.162242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-762f9ea0-6e86-43ee-98c6-1a65f07c1559 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.163756 May 23 16:15:36.162814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1716/3539] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:36.234708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:36.242949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a4e40c49-ab3d-4bd2-9f54-d5a0f68e446e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151148 May 23 16:15:36.243547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1824/3540] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:37.179215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:37.179853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:37.186252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:37.186697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:37.187474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:37.187907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:37.259994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:37.260610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:37.267671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:37.268117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:37.268873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:37.269305 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:37.365138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:37.373347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62a11eef-55c0-4ec1-b2d6-b555a2b48ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.198034 May 23 16:15:37.374008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1717/3541] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:37.428049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:37.436108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b2880f6-3030-4501-bec5-ab716139dc32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.179922 May 23 16:15:37.436621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1825/3542] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:38.390825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:38.391472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:38.398150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:38.398591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:38.399382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:38.399801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:38.404280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:38.453187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:38.453797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:38.460328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:38.460755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:38.461506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:38.461938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:38.569028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:38.577119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f6b03006-3ccb-4bb4-b1cb-597d988297a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.190533 May 23 16:15:38.577694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1718/3543] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:38.613198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:38.621084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-52b58efe-2447-45c6-a492-36a478662d7e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.171977 May 23 16:15:38.621663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1826/3544] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:39.594336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:39.594973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:39.601635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:39.602090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:39.602886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:39.603348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:39.637508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:39.638256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:39.644738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:39.645255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:39.646334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:39.646769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:39.783299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:39.791236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2549a6a4-b3fa-4b67-b107-ec5976b1ab9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.200930 May 23 16:15:39.791819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1719/3545] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:39.810511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:39.818455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4bc2c5ee-7d04-41ac-94f1-32a6311e89d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.184674 May 23 16:15:39.819017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1827/3546] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:40.809545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:40.810252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:40.816619 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:40.817100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:40.817859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:40.818293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:40.835314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:40.839199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:40.842258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:40.842706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:40.843493 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:40.843928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:41.014396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:41.017993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:41.022735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7fb3f760-7caf-4c4a-b4a5-574ff7f7eb77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217513 May 23 16:15:41.023327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1720/3547] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:41.026256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc393c08-24c4-4db7-832c-2a6ad5ee8be4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.194804 May 23 16:15:41.026814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1828/3548] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:42.039868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:42.040486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:42.042328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:42.042949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:42.047156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:42.055632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:42.055632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:42.055632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:42.055978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:42.055978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:42.055978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:42.055978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:42.245087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:42.247974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:42.253946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-99534237-d456-472f-958e-f6f9f0219fc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217933 May 23 16:15:42.254520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1721/3549] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:42.256345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-418befb1-363a-42bf-a3ce-2a5596312d25 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217828 May 23 16:15:42.256902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1829/3550] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:43.272123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:43.272774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:43.273962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:43.274581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:43.279274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:43.279732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:43.281084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:43.281521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:43.281629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:43.282083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:43.282843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:43.283310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:43.467461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:43.471944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:43.476205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2b6f4c84-9fb2-407c-82d1-e159653ceb01 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208164 May 23 16:15:43.476778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1722/3551] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:43.479901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aba890b5-3e02-47b3-878f-36b335a0c4ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210293 May 23 16:15:43.480462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1830/3552] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:44.494100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:44.494695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:44.497526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:44.498148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:44.501482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:44.501930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:44.502694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:44.503189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:44.504738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:44.505166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:44.505923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:44.506359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:44.710091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:44.714151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:44.720669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a80e4865-10b7-4f69-b786-b127059b8b21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.230573 May 23 16:15:44.721257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1723/3553] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:44.722965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b6411a60-d6bc-4252-a8ff-59566b0317e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.229537 May 23 16:15:44.723671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1831/3554] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:45.738394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:45.739013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:45.740103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:45.740711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:45.745666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:45.746138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:45.746886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:45.747075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:45.747356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:45.747549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:45.748314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:45.748722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:46.189484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:46.193356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:46.197413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-469be479-ffb5-4257-b761-10fc732bde46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.463391 May 23 16:15:46.197979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1724/3555] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:46.201176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5b5b245-ccd5-4dee-974d-effedb167642 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.464857 May 23 16:15:46.201734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1832/3556] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:47.215222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:47.215845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:47.217488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:47.218102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:47.222415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:47.222868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:47.223668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:47.224108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:47.224416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:47.224863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:47.225604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:47.226036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:47.417481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:47.421536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:47.426228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72a275fb-6eb4-420c-8507-40380453a3d9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214802 May 23 16:15:47.426791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1725/3557] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:47.429960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb21c3cd-005a-4f50-bf7b-61f10f16c4e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216272 May 23 16:15:47.430584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1833/3558] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:48.444230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:48.444857 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:48.446461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:48.447071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:48.451447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:48.451907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:48.452677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:48.453117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:48.453551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:48.454017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:48.454766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:48.455228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:48.640749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:48.647742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:48.648922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cd2a4719-ce32-4a23-a0e0-8f4e65d7efaf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.206275 May 23 16:15:48.649502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1834/3559] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:48.655931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a3b88d88-8828-4ad1-802a-512d7d27bd9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215627 May 23 16:15:48.656476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1726/3560] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:49.665773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:49.666415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:49.672628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:49.672853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:49.673293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:49.673480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:49.673813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:49.674245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:49.679878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:49.680329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:49.681096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:49.681533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:49.863474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:49.870113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:49.872249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6424f83e-bb49-4df3-85ca-60f00ed3dbbc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210240 May 23 16:15:49.872836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1835/3561] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:49.878541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55ee2875-d3f4-49cb-a840-c9912c6d576e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209397 May 23 16:15:49.881298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1727/3562] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:50.889210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:50.889831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:50.896645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:50.896825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:50.897278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:50.897405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:50.897915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:50.898353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:50.899710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:50.904401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:50.905184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:50.905943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:50.906376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:51.091288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:51.095889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:51.099714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-879f7fb8-cd28-4c7c-867f-d21e3dbcfb75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214414 May 23 16:15:51.100315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1836/3563] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:51.103751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a5122d2-dfde-4c4f-a7a1-14483d423d0e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211210 May 23 16:15:51.104305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1728/3564] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:52.117012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:52.117648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:52.120006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:52.121345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:52.121961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:52.124722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:52.125162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:52.126508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:52.126940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:52.128894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:52.133957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:52.133957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:52.133957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:52.134260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:52.318743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:52.322293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:52.329465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-81b09c2f-278c-42e0-a48c-4e5f7465e784 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216315 May 23 16:15:52.330058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1837/3565] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:52.330468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75a633a4-52d8-4205-9cf8-f42debc35545 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213007 May 23 16:15:52.331033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1729/3566] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:53.348834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:53.349578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:53.349686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:53.350356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:53.357567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:53.358011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:53.358208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:53.358667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:53.358863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:53.359222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:53.361894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:53.362348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:53.557952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:53.561909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:53.566627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb58df7f-d549-4378-be07-f5d4c8f52b33 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.222819 May 23 16:15:53.567221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1838/3567] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:53.570303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5657c1dc-2394-48dc-80f7-dd2adcfa18e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225382 May 23 16:15:53.570856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1730/3568] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:54.584389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:54.584927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:54.587389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:54.588013 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:54.591541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:54.591987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:54.592780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:54.593223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:54.594964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:54.595444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:54.596196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:54.596630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:54.785540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:54.789340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:54.793984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d98b0111-e7ad-4cdc-bc7e-a2ace4557ea3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213536 May 23 16:15:54.794653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1839/3569] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:54.797324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a6cf7658-79f7-4c31-8c11-bc8e1fde27d6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213669 May 23 16:15:54.797874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1731/3570] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:55.811200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:55.811829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:55.813632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:55.814243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:55.818387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:55.818844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:55.819711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:55.820094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:55.820244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:55.820545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:55.821296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:55.821736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:56.007975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:56.014610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:56.016584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36bbe6be-7c1d-4367-85cc-d80e45a01c3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208806 May 23 16:15:56.017247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1840/3571] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:56.025123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d76680d-b6ea-425c-9a2f-5650024eef77 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215482 May 23 16:15:56.025995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1732/3572] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:57.034439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:57.035076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:57.041492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:57.041838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:57.042280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:57.042461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:57.042672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:57.043119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:57.049821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:57.050273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:57.051038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:57.051508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:57.236473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:57.239566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:57.244939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-71c5f101-fe5a-4da6-b1f5-cabfb85361b0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214836 May 23 16:15:57.245593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1841/3573] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:57.248551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb8db49b-b253-48bb-805a-ba47828d99ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210456 May 23 16:15:57.249122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1733/3574] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:58.347570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:58.347570 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:58.349128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:58.349128 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:58.349628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:58.349628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:58.349628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:58.349628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:58.350015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:58.350015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:58.350015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:58.350015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:58.465707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:58.468808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:58.472187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:15:58.473700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9d2a2e4-d6a4-41ee-a1d9-96eb3d3774b1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215374 May 23 16:15:58.474282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1842/3575] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:58.477126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c945a504-43a4-41fc-b53e-feb096803761 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215590 May 23 16:15:58.483590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1734/3576] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:15:59.491301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:59.491876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:59.498245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:15:59.498722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:59.498722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:59.498953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:15:59.499465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:59.499901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:59.505891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:59.506337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:59.507090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:15:59.507545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:15:59.738777 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:59.742773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:15:59.747156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c13d854b-531e-4cd2-bb10-cb46ec3de17a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.259723 May 23 16:15:59.747732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1843/3577] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:15:59.751464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-197ab434-8d0f-4cb4-9656-c0b5250f5772 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.257268 May 23 16:15:59.752098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1735/3578] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:15:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:00.765227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:00.765849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:00.767351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:00.768278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:00.772739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:00.773187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:00.773950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:00.774381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:00.777205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:00.777656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:00.778427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:00.778907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:00.964295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:00.971467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:00.972807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fb5e127b-e8e3-4286-b611-8b43b76b770e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211696 May 23 16:16:00.973383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1844/3579] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:00.979499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-05c299a0-0cc6-4f65-ad1f-c7e825a3ae99 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216057 May 23 16:16:00.980059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1736/3580] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:01.990590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:01.991240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:01.996901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:01.997512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:01.997883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:01.998342 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:01.999124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:01.999573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:02.004523 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:02.004970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:02.005720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:02.006158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:02.194041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:02.198347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:02.202122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9524affa-b880-4438-8c8f-14bf2e0f4b54 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215384 May 23 16:16:02.202941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1845/3581] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:02.206433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9cc317e0-8cd8-49a6-93f2-75ccd92857d2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213437 May 23 16:16:02.206985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1737/3582] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:03.219678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:03.220281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:03.223891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:03.224521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:03.226699 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:03.227172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:03.227932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:03.228365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:03.231353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:03.231828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:03.232604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:03.233049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:03.420878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:03.424545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:03.429137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b2bc87ad-6883-49b3-a05d-d3f01a716641 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213924 May 23 16:16:03.429760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1846/3583] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:03.432722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d3ac001a-769b-41f4-b4f2-a72a11a0b0ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212552 May 23 16:16:03.433383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1738/3584] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:04.445706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:04.446308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:04.449426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:04.450040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:04.452712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:04.453170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:04.453937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:04.454365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:04.456677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:04.457186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:04.457958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:04.458396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:04.651383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:04.655325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:04.659819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d75da6fa-2e39-4714-b49e-061c84ea080c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217919 May 23 16:16:04.660392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1847/3585] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:04.663840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ac1f58e6-7646-4712-93a0-adb84dbf3217 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218303 May 23 16:16:04.664395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1739/3586] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:05.677228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:05.677852 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:05.681019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:05.681632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:05.684371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:05.684829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:05.685594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:05.686039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:05.688594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:05.689040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:05.689788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:05.690220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:05.876982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:05.881101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:05.884935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-97fecdb6-cf95-4f45-bb3d-d8494f7163aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211536 May 23 16:16:05.885700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1848/3587] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:05.889158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b5577ef-d28b-47bf-893a-7d173d568fd5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212005 May 23 16:16:05.889718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1740/3588] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:06.902384 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:06.903033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:06.905804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:06.906426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:06.909612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:06.910064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:06.910824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:06.911292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:06.912881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:06.913331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:06.914091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:06.914519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:07.101329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:07.107987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:07.110296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5e640fb-dc3c-4afd-b5ee-ec5d301dd077 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211225 May 23 16:16:07.110889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1849/3589] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:07.116468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1983ecb8-440b-4c7b-bc9d-8ecf9658223b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214414 May 23 16:16:07.117027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1741/3590] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:08.128160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:08.128861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:08.133317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:08.133963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:08.135364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:08.135813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:08.136568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:08.136999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:08.140327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:08.141286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:08.142036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:08.142465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:08.336582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:08.339901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:08.344878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-99f81884-0e8d-4027-9956-4f57db1a22ae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220539 May 23 16:16:08.345456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1850/3591] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:08.347994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-403a2fe9-15d6-4671-8702-8717d0eae011 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218428 May 23 16:16:08.348547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1742/3592] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:09.362411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:09.363022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:09.365162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:09.365786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:09.369710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:09.370156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:09.370908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:09.371366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:09.372189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:09.372625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:09.373383 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:09.373824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:09.555893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:09.563694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-353c8fca-3d44-4804-9a90-7d297f2356a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.205769 May 23 16:16:09.564489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1851/3593] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:09.569300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:09.577496 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f1c2da73-7ce9-4b75-87fa-0809bf369f44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216178 May 23 16:16:09.578549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1743/3594] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:10.580739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:10.581319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:10.587756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:10.588203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:10.588952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:10.589385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:10.595323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:10.595938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:10.602435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:10.602878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:10.603672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:10.604103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:10.790164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:10.794274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:10.799266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b1186f6c-a9e4-4fae-bb69-ef93bfe2a002 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.222188 May 23 16:16:10.802363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1852/3595] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:10.802814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6564b42-dea7-4153-866e-47ecdb0bd260 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211894 May 23 16:16:10.803413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1744/3596] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:11.818615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:11.819263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:11.820093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:11.820721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:11.825788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:11.826239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:11.826999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:11.827210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:11.827464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:11.827667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:11.828432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:11.828873 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:12.012303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:12.015677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:12.027659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6f087f5c-0b36-4101-8bdc-415089119dbb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.206356 May 23 16:16:12.027659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1853/3597] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:12.027900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:12.032437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-30aa4668-5ecc-4583-ab2b-5a2889132f38 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215809 May 23 16:16:12.033056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1745/3598] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:13.038048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:13.038657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:13.045172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:13.045623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:13.046369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:13.046800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:13.048725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:13.049341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:13.055706 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:13.056155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:13.056904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:13.057334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:13.235955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:13.243048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-925a075e-57e2-4563-8277-720f88106791 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209165 May 23 16:16:13.244349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1854/3599] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:13.246153 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:13.254184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5a665b80-23b1-4dc0-9ab0-31812e6a69a3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209291 May 23 16:16:13.254737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1746/3600] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:14.260291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:14.265325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:14.267995 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:14.268473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:14.269224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:14.269652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:14.271843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:14.272469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:14.278981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:14.279516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:14.280357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:14.280803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:14.461024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:14.464913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:14.469010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-abb28f3d-7e53-40f3-b467-ee1fa0925ac3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212531 May 23 16:16:14.469576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1855/3601] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:14.473085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-17da0a0d-00f9-48d6-9d57-80f47431ecba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.205857 May 23 16:16:14.473650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1747/3602] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:15.486523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:15.487198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:15.490497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:15.491132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:15.493791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:15.494275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:15.495046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:15.495509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:15.497499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:15.497947 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:15.498700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:15.499165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:15.688135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:15.691590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:15.696930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c4f3ba62-b637-4d5f-bc1d-3f69c9b558b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214817 May 23 16:16:15.697595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1856/3603] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:15.699585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35a2216f-7636-422f-8af9-fd18f3a47f0f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213393 May 23 16:16:15.700178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1748/3604] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:16.714072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:16.714712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:16.716799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:16.717411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:16.721207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:16.721652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:16.722411 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:16.722864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:16.723738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:16.724192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:16.724942 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:16.725373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:16.915499 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:16.918544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:16.924301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca623824-73d2-45ce-b6db-2f5d4929f682 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214223 May 23 16:16:16.924880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1857/3605] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:16.926953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fe459c2f-ed75-4e29-81bb-873462a55be2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213926 May 23 16:16:16.927735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1749/3606] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:17.941496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:17.942100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:17.943712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:17.944337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:17.948527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:17.948970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:17.949718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:17.950171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:17.950733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:17.951237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:17.952000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:17.952438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:18.139928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:18.144567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:18.148694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dc71b407-38d0-4541-a6b2-921c75c510fb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211091 May 23 16:16:18.149267 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1858/3607] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:18.152884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6362b754-d047-4a6c-92fe-e503b5df59a4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212959 May 23 16:16:18.153479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1750/3608] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:19.165684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:19.166296 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:19.169600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:19.170217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:19.173014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:19.173464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:19.174250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:19.174680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:19.176570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:19.177158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:19.177781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:19.178217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:19.195705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:19.365685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:19.369588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:19.373669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0a459cb-1c36-4866-b056-95c85310010d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212074 May 23 16:16:19.374471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1859/3609] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:19.377529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a543b81f-a64b-42fc-aa78-bee78e18fecd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211695 May 23 16:16:19.378083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1751/3610] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:20.389643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:20.390255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:20.394038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:20.394697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:20.398035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:20.398483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:20.399271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:20.399704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:20.401427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:20.401886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:20.402654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:20.403110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:20.590068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:20.593004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:20.597949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3611f4ff-616a-447c-a435-1d0fc15228b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212130 May 23 16:16:20.598518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1860/3611] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:20.602599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-243e9d75-bfb5-4d9c-985d-8df95b1d7a8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211490 May 23 16:16:20.602599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1752/3612] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:21.615286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:21.615886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:21.617732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:21.618344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:21.620679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:21.622611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:21.623068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:21.623967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:21.624390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:21.625293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:21.625741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:21.626537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:21.626971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:21.815875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:21.819772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:21.823934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7e24ee49-5e9a-4ebc-bb98-6e928fc7cca2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212698 May 23 16:16:21.824513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1861/3613] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:21.827780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1d6ee037-3f2d-48dd-a078-e3a464256607 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213932 May 23 16:16:21.828331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1753/3614] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:22.841262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:22.841887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:22.844227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:22.844936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:22.845510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:22.851906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:22.855527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:22.855661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:22.855791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:22.855883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:22.855994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:22.856091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:22.856091 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:23.053729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:23.057698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:23.061913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1c901228-0c3e-4485-933f-8270dd90a77f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220664 May 23 16:16:23.062493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1754/3615] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:23.066359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-727923e8-e628-410b-ae91-b18dd9a380dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.229101 May 23 16:16:23.066935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1862/3616] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:24.080214 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:24.080811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:24.083622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:24.084237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:24.087288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:24.087740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:24.088495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:24.088924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:24.091058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:24.091541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:24.092297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:24.092736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:24.281818 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:24.286429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:24.293047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-98ae4e13-2855-42e2-b582-b53d28bbde62 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217200 May 23 16:16:24.294404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-449e46ba-febf-4d87-a97c-d46225132177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215263 May 23 16:16:24.296426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1863/3617] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:24.296586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1755/3618] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:25.310796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:25.311450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:25.315367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:25.315989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:25.318178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:25.318603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:25.319399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:25.319836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:25.322452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:25.322906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:25.323705 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:25.324149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:25.752878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:25.756070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:25.761581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1d95a898-1f63-4556-8578-4fddf8d24269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.454668 May 23 16:16:25.762158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1864/3619] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:25.764184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cfbe6412-6e19-41fe-b330-edfb6ac2dd0e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.452685 May 23 16:16:25.767534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1756/3620] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:26.780379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:26.781002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:26.781342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:26.782016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:26.788102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:26.788556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:26.788976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:26.789318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:26.789431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:26.789757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:26.790160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:26.790595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:26.980627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:26.982829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:26.990794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-91236772-c8c6-450f-9f8b-3b1e3f0fb0b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214354 May 23 16:16:26.991394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1865/3621] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:26.992450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ed2eda27-563c-4952-a5a9-d761ded1df68 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214757 May 23 16:16:26.993300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1757/3622] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:28.008198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:28.008807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:28.009353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:28.009970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:28.015324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:28.015772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:28.016322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:28.016521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:28.017386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:28.017601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:28.018154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:28.018588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:28.204586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:28.211160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:28.212889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a659ab1e-dfa1-4e25-8cd6-7f88dd1f3da1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208533 May 23 16:16:28.213479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1866/3623] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:28.219355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5679c2f3-ac1f-43b2-b480-17960c3e93d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213654 May 23 16:16:28.219935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1758/3624] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:29.229860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:29.230633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:29.236483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:29.237111 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:29.237479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:29.237934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:29.238702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:29.239161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:29.243450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:29.243906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:29.244674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:29.245113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:29.428456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:29.431525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:29.436479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-60adb97f-5f13-4a4e-858a-bd5d1bee3294 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210479 May 23 16:16:29.437151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1867/3625] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:29.439994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bcd7a3e3-fce2-4d02-80a4-bf50c07cdc34 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207819 May 23 16:16:29.440553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1759/3626] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:30.454193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:30.454804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:30.456051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:30.456661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:30.461364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:30.461818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:30.462587 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:30.463027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:30.463190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:30.463531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:30.464295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:30.464723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:30.655944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:30.660013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:30.664051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8780d77f-20b7-46c1-9e73-494bb6a151ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211789 May 23 16:16:30.664618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1760/3627] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:30.668015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bd11d031-bcbf-45b5-b7b3-15cf5c706645 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217811 May 23 16:16:30.668576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1868/3628] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:31.681434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:31.681990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:31.684598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:31.685222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:31.688959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:31.689421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:31.690170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:31.690598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:31.692452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:31.692909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:31.693677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:31.694131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:31.879878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:31.886400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:31.887746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-988ecc34-0ff7-4fc1-bba7-9d97fad4889b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210279 May 23 16:16:31.888318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1761/3629] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:31.894605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8163754-1fdb-46ae-b1dd-4935c9e270e8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213605 May 23 16:16:31.895182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1869/3630] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:32.905210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:32.905937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:32.912074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:32.912725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:32.915127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:32.915679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:32.916462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:32.916897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:32.919246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:32.919706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:32.920561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:32.921006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:32.943174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:33.139820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:33.147992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:33.148871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5c2f9946-ea48-4d80-986c-79c1cf409e6a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.241224 May 23 16:16:33.149452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1870/3631] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:33.156080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-364a9ba4-e772-4056-ae27-4c05fa09f9f0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.254625 May 23 16:16:33.156585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1762/3632] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:34.166894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:34.167537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:34.174179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:34.174638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:34.175432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:34.175875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:34.176198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:34.176943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:34.185027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:34.185481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:34.186252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:34.186862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:34.356739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:34.365273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9a2e910-e256-4cd9-b6fe-23ada57b21b3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.201672 May 23 16:16:34.365876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1871/3633] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:34.379236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:34.388694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-266cc62c-9045-4bf8-bc7b-21c2b16b1416 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216317 May 23 16:16:34.389263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1763/3634] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:35.382268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:35.382875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:35.389302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:35.389748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:35.390526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:35.390961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:35.406022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:35.406593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:35.413506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:35.413964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:35.414722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:35.415191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:35.596834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:35.601007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:35.605520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-66f11a00-5e26-4441-814b-13f0d118ea2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.227045 May 23 16:16:35.606101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1872/3635] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:35.609361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84530966-3b20-4f8b-8d57-8c54de87e738 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207358 May 23 16:16:35.609940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1764/3636] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:36.624572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:36.625126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:36.626665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:36.627338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:36.631775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:36.632234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:36.633008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:36.633451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:36.633670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:36.634113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:36.634868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:36.635333 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:36.830165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:36.833502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:36.838176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-87b4ef95-1926-43fd-8e32-b19f3fc52f30 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217784 May 23 16:16:36.838760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1873/3637] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:36.843274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e4823b12-8dbf-4418-978d-491f9ab6e316 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.221366 May 23 16:16:36.843842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1765/3638] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:37.858053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:37.858675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:37.860308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:37.860927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:37.865298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:37.865746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:37.866505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:37.866938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:37.867526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:37.868004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:37.868761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:37.869198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:38.072386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:38.075499 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:38.081207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a53336cc-f135-4e72-a058-79b08f2f7df8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.227052 May 23 16:16:38.081789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1874/3639] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:38.083916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2c5deeff-9514-4888-b08a-8638b7685710 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.227430 May 23 16:16:38.084484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1766/3640] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:39.100586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:39.101252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:39.101921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:39.102591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:39.107760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:39.108208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:39.108963 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:39.109088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:39.109392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:39.109530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:39.110180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:39.110631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:39.296852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:39.304874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0b13c0db-2266-44c6-91f9-7755678f4a8b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208137 May 23 16:16:39.305460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1767/3641] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:39.311202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:39.319820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-44cf8a43-db7b-4d49-b4b8-943a71eea990 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.222976 May 23 16:16:39.320389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1875/3642] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:40.322756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:40.323403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:40.331634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:40.331634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:40.331864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:40.332190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:40.336304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:40.336442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:40.337105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:40.344612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:40.349436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:40.349436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:40.349436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:40.536500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:40.539876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:40.546048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-074006a0-b00b-49f5-81d6-307cd71602cc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213630 May 23 16:16:40.546638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1876/3643] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:40.547953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4ed66015-5bd4-4e66-93eb-e635a16e180c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.229033 May 23 16:16:40.548520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1768/3644] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:41.563410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:41.579533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:41.579695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:41.579695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:41.580279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:41.580279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:41.580490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:41.580490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:41.580711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:41.580711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:41.580904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:41.580904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:41.767279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:41.770448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:41.776769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af663522-00f4-4c66-bb82-ff4e2b728dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217370 May 23 16:16:41.783990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1877/3645] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:41.784151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d14f2ed1-eeca-42cc-92da-66482c32c372 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218469 May 23 16:16:41.784151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1769/3646] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:42.794376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:42.794988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:42.796373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:42.796998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:42.801478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:42.801919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:42.802667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:42.803114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:42.803265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:42.803725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:42.804494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:42.804936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:42.992220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:42.996164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:43.000235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc222179-bbc9-42bb-b3b8-c014533705e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209623 May 23 16:16:43.001520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1878/3647] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:43.005034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-879be8f3-3bfb-4aa7-9fac-aa1158f9efe8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212631 May 23 16:16:43.005601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1770/3648] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:44.018037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:44.018648 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:44.022157 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:44.022773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:44.025480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:44.025925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:44.026674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:44.027121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:44.029224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:44.029680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:44.030439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:44.030914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:44.220335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:44.224516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:44.228537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-13f4f68b-7775-4fbc-b742-db5990684bd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214627 May 23 16:16:44.229797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1879/3649] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:44.233018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb589aa7-2f7d-4ad9-b888-f5773dfdb180 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214819 May 23 16:16:44.237242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1771/3650] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:45.245781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:45.246400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:45.249413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:45.250030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:45.253515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:45.253962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:45.254719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:45.255177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:45.257122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:45.257571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:45.258325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:45.258759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:45.452344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:45.455743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:45.460469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7eac277-eb5b-44fb-bcea-90b3e16eece1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218513 May 23 16:16:45.461040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1880/3651] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:45.463814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83b096fa-e114-41df-9792-d7ac739a8670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218158 May 23 16:16:45.464375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1772/3652] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:46.478485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:46.479121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:46.479828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:46.480452 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:46.486078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:46.486456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:46.486792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:46.487234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:46.487353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:46.487791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:46.488058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:46.488481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:46.677794 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:46.680925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:46.685864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-997bf40b-e0d8-41db-9bfa-0d497839341b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209798 May 23 16:16:46.686440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1773/3653] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:46.689408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1aa90f6-f40c-4042-b2a7-5cc5047d2468 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214850 May 23 16:16:46.689967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1881/3654] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:47.703127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:47.703694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:47.705953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:47.706602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:47.710690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:47.711170 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:47.711945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:47.712384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:47.712910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:47.713354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:47.714108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:47.714541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:47.902218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:47.905457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:47.911492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9d2c7ce-e318-44cf-8c7c-20fbb96c293e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209787 May 23 16:16:47.912076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1882/3655] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:47.914246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aba0a675-e80b-4e84-b333-fbba8e078c2e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215160 May 23 16:16:47.914812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1774/3656] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:48.929292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:48.929918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:48.930756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:48.931404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:48.936483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:48.936937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:48.937708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:48.937859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:48.938133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:48.938267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:48.938877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:48.939348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:49.375379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:49.378868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:49.383719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a8790d9b-2885-4b56-86b5-4bc139ab3ea2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.456724 May 23 16:16:49.384292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1775/3657] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:49.386786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-620de6ae-581a-468f-b4af-eec60b863986 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.461453 May 23 16:16:49.387410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1883/3658] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:50.401202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:50.401844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:50.404185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:50.404802 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:50.408893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:50.409921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:50.410690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:50.411149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:50.411650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:50.412101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:50.412857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:50.413427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:50.592778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:50.601509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55cd92c9-5b45-4459-bdb6-a5f575b9dea4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.204156 May 23 16:16:50.602086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1776/3659] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:50.603943 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:50.611873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-37cc84b9-cd93-4f2d-b52e-5dc1b86b71b6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211495 May 23 16:16:50.612430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1884/3660] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:51.618698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:51.619348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:51.626023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:51.626470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:51.627327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:51.627759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:51.628929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:51.629551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:51.636048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:51.636501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:51.637276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:51.637698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:51.824448 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:51.827580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:51.833160 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2eeca4c7-21e4-4396-ac0c-8d3c3e008bdf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218594 May 23 16:16:51.833732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1777/3661] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:51.836455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-992d117f-a98c-4843-b160-cda8d4a712eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211488 May 23 16:16:51.837020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1885/3662] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:52.850015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:52.850633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:52.853022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:52.853303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:52.853924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:52.857782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:52.858232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:52.859002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:52.859469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:52.867260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:52.867260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:52.867260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:52.867260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:53.056379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:53.064955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-333b9aab-5106-4f21-9030-89efb50303ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218749 May 23 16:16:53.065529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1778/3663] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:53.067971 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:53.075976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-def3a66e-19f5-4e17-9d03-cfad20dcb127 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.226528 May 23 16:16:53.076537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1886/3664] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:54.088282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:54.089097 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:54.093862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:54.094518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:54.096872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:54.097489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.097946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.098724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.099184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.101593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.102037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.102847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.103769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.107911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:54.299327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:54.302450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:54.307808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dae72172-927f-4c12-82e8-fe5411d86d93 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218038 May 23 16:16:54.308744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1887/3665] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:54.313647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f770bb4f-db3e-4304-99c9-05dcba430663 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.230672 May 23 16:16:54.313804 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:54.314372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1779/3666] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:54.314516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:54.324700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.325141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.325898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:54.326354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:54.351858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70fc2406-29b8-4625-8f5a-a5dc722f8efe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/ips" status: 200 len: 150 microversion: 2.1 time: 0.042075 May 23 16:16:54.352355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1888/3667] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 16:16:54 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:16:55.325336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:55.325972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:55.330523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:55.331149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:55.332840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:55.333292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:55.334441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:55.334603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:55.337781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:55.338235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:55.339038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:55.339498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:55.534124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:55.538013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:55.542192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1751990b-36ca-456c-8956-72483f5fb769 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220783 May 23 16:16:55.542771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1780/3668] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:55.546248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-beb45092-f99e-4f87-b968-7b60a4e8b9b2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219962 May 23 16:16:55.546809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1889/3669] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:56.559493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:56.560039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:56.562685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:56.563832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:56.567131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:56.567574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:56.568328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:56.568759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:56.570266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:56.570724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:56.571520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:56.571965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:56.761291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:56.764326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:56.769953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-81acc3e4-1258-4198-b467-0d09c411f442 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214410 May 23 16:16:56.770760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1781/3670] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:56.773090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76614105-e063-4239-b630-51d14b94384c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214130 May 23 16:16:56.773649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1890/3671] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:57.518733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:57.519166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:57.525414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.525861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.526610 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.527041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.549615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d2b2626f-2799-42df-968e-1b981eefdf38 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/ips" status: 200 len: 151 microversion: 2.20 time: 0.034811 May 23 16:16:57.550184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1782/3672] 10.222.0.32 () {60 vars in 1288 bytes} [Mon May 23 16:16:57 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/ips => generated 151 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:16:57.787169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:57.787712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:57.790073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:57.790687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:57.794375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.794836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.795639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.796079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.797012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.797459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.798215 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:57.798646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:57.991834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:57.994611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:58.000190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-26668aef-35f2-44be-909a-7b482912c6ae tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213840 May 23 16:16:58.001298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1783/3673] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:16:58.003109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6cc98571-a64c-4337-b429-06c61d2a1d21 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219820 May 23 16:16:58.003750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1891/3674] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:59.018834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:59.019469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:59.021277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:16:59.022143 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:16:59.026270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:59.026723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:59.027516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:59.027954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:59.028815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:59.029256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:59.030010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:16:59.030442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:16:59.220984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:59.223927 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:16:59.229189 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-11800a3f-090d-4830-8f4d-44fabf686586 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214625 May 23 16:16:59.229953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1784/3675] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:16:59.233317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5e9c3d0b-b8c2-4621-ac0a-c301654b916c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215922 May 23 16:16:59.233990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1892/3676] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:16:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:00.245959 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:00.246581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:00.249429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:00.250202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:00.253891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:00.254339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:00.255121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:00.255563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:00.257298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:00.257751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:00.258515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:00.258954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:00.259549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:00.448359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:00.451644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:00.456521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-911e7ef8-8e12-4496-b544-631e249b5027 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214480 May 23 16:17:00.457282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1785/3677] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:00.459881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1a031bc-754c-462f-84c0-dfe6ad3790a0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214257 May 23 16:17:00.460447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1893/3678] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:01.473984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:01.474613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:01.476707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:01.477333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:01.481464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:01.481922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:01.482700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:01.483194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:01.483811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:01.484264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:01.485074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:01.485461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:01.678240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:01.682073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:01.686534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4d918af8-572d-42b2-88c0-c0563c1a38c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217057 May 23 16:17:01.687127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1786/3679] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:01.689973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da029634-6e8e-446a-aceb-7c299b0d89b2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217257 May 23 16:17:01.690591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1894/3680] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:02.703401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:02.704466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:02.706630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:02.707275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:02.711073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:02.711545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:02.712304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:02.712737 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:02.714050 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:02.714498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:02.715282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:02.715724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:02.907972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:02.911893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:02.916220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-57a6c630-c572-4d70-81a7-0c61d05af182 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216751 May 23 16:17:02.916877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1787/3681] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:02.919666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77f28f68-88f6-4766-988d-931c11e89fb0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217366 May 23 16:17:02.920223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1895/3682] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:03.933098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:03.933805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:03.937071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:03.937767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:03.940768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:03.941224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:03.941985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:03.942414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:03.944418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:03.944916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:03.945759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:03.946208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:04.137594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:04.140798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:04.146871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0bd6f6c7-8635-45f5-947c-aa059175b99a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217455 May 23 16:17:04.147485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1788/3683] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:04.149556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-be2b5d56-7551-48d0-9a31-fdf38ce2a730 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216214 May 23 16:17:04.150132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1896/3684] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:05.167459 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:05.168004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:05.168377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:05.168668 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:05.175258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:05.177734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:05.178369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:05.178369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:05.178601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:05.178601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:05.178807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:05.178807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:05.364710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:05.371918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:05.372798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0e00b20a-3e3f-4e2e-b427-4674aba6257d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208635 May 23 16:17:05.373375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1897/3685] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:05.380625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d516920e-a3ec-45c5-9886-f502cb5f69b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217057 May 23 16:17:05.381146 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1789/3686] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:06.389875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:06.390413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:06.396477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:06.397648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:06.398063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:06.398297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:06.398460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:06.398891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:06.404745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:06.405192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:06.405958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:06.406394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:06.593634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:06.597782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:06.601568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-09b4f9c4-2604-4fb9-bdbf-839189d1117c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215728 May 23 16:17:06.602130 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1898/3687] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:06.605729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-259e2813-ea07-4c42-86ce-dffe6e124740 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212176 May 23 16:17:06.606291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1790/3688] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:07.619158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:07.619765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:07.622726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:07.623373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:07.626292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:07.626743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:07.627539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:07.627983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:07.629511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:07.629955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:07.630709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:07.631167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:07.818956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:07.822134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:07.826863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5fa6044d-91e9-459f-b2ab-e28dade88bde tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211561 May 23 16:17:07.827650 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1899/3689] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:07.830575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e340f344-00dc-40e9-b1c8-00ad2380eaf6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211570 May 23 16:17:07.831272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1791/3690] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:08.844891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:08.845513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:08.847937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:08.848572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:08.852092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:08.852540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:08.853311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:08.853749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:08.855216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:08.855679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:08.856465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:08.856908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:09.047498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:09.054524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:09.058251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-290fde16-75f0-41f6-a698-b6d8e072ee17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217836 May 23 16:17:09.058925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1900/3691] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:09.062469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-702dbb7e-b4eb-47db-a46b-1feb589955dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219071 May 23 16:17:09.063558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1792/3692] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:10.075996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:10.076542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:10.083556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:10.084008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:10.084800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:10.085075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:10.085513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:10.085722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:10.092190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:10.092641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:10.093403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:10.093969 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:10.281206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:10.288774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:10.289902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-45e6d2fa-7457-4eda-a36c-5351f63e7480 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217856 May 23 16:17:10.290511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1901/3693] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:10.297047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9a1cffb-8fc2-4105-8acf-7874941dcf3c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215993 May 23 16:17:10.297639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1793/3694] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:11.306926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:11.307571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:11.313589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:11.314128 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:11.314281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:11.314576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:11.315872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:11.316307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:11.321276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:11.321727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:11.322489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:11.322929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:11.505617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:11.509995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:11.513921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-33ed1878-57a2-4ba2-87af-03b6f0cd6ce3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210828 May 23 16:17:11.514502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1902/3695] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:11.518035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77de00b9-bf71-46c5-9c2a-8695be47ec4e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208213 May 23 16:17:11.519119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1794/3696] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:12.531954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:12.532523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:12.536649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:12.537271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:12.539281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:12.539743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:12.540684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:12.541123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:12.543851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:12.544307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:12.545064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:12.545500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:12.730622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:12.737925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:12.739253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a599300b-edfc-46da-bb51-eca1b97bb6ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211188 May 23 16:17:12.740059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1903/3697] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:12.746073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e213fc9-8b12-4168-b29c-65c4df41903e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213236 May 23 16:17:12.746630 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1795/3698] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:13.756533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:13.757150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:13.761885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:13.762492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:13.763684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:13.764134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:13.764888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:13.765318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:13.769608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:13.769800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:13.770060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:13.770865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:13.771358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:13.957996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:13.961016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:13.966300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-46fd2452-b521-4dbc-aa61-1893e916a71f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214168 May 23 16:17:13.966898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1904/3699] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:13.969129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6403417c-58c0-4852-8f7c-56cf7ce2feb6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211032 May 23 16:17:13.969685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1796/3700] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:14.982811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:14.983451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:14.985487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:14.986118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:14.989942 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:14.990392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:14.991170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:14.991609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:14.992442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:14.992901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:14.993664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:14.994129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:15.184667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:15.188642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:15.193602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-46c382f4-c6b2-4cdd-bc76-d419a91c5b2b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214575 May 23 16:17:15.195207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1905/3701] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:15.197845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6e6c4924-0c97-4d74-b19e-d49b8351700c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216192 May 23 16:17:15.198403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1797/3702] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:16.211821 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:16.212440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:16.215073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:16.215711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:16.218962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:16.219440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:16.220198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:16.220627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:16.223130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:16.223709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:16.224544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:16.224989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:16.409637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:16.416409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:16.417622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-143e9430-2972-4608-91ce-27d34f0f33ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210167 May 23 16:17:16.418292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1906/3703] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:16.424453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ae006a40-7e2d-4a23-b49b-979d7a53dcce tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213185 May 23 16:17:16.425011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1798/3704] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:17.434859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:17.435505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:17.441839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:17.442322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:17.442451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:17.442581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:17.443263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:17.443706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:17.448915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:17.460855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:17.460855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:17.460855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:17.581937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:17.590294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0cac4ab9-922a-4c38-aa2b-60cd8a4c0cd7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.159204 May 23 16:17:17.590893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1907/3705] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:17.740429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:17.744230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f7d38b8-bedf-446e-9a4d-8ea5a06be633 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.306817 May 23 16:17:17.744765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1799/3706] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:18.607378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:18.607920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:18.615079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:18.615558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:18.616322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:18.616755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:18.743293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:18.751070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4b43e6c5-67fb-40f0-9ca4-8132da293188 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.147681 May 23 16:17:18.751652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1908/3707] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:18.760855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:18.761473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:18.767809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:18.768259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:18.769005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:18.769435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:18.894533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:18.902615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8309028d-0435-43de-ab43-f3500a4e54f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.145579 May 23 16:17:18.903179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1800/3708] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:19.767911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:19.769111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:19.775838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:19.776281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:19.777025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:19.777455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:19.903412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:19.911586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b40e5de2-135c-4a9f-9460-689b1a736ab3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.147535 May 23 16:17:19.912156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1909/3709] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:19.920076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:19.920693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:19.927738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:19.928187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:19.928946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:19.929384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:20.055832 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:20.059194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:20.066480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-18f75c62-9f1a-4d14-84ad-c067abd34567 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.150155 May 23 16:17:20.067257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1801/3710] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:20.928675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:20.929285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:20.935771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:20.936214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:20.936959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:20.937389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:21.064721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:21.073041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8fda024c-24c1-4078-9d2c-e4389ee8b758 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.148226 May 23 16:17:21.073600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1910/3711] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:21.084904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:21.085525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:21.092344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:21.092804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:21.093566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:21.094003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:21.223754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:21.231629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39fa8c9b-4e82-4740-8aff-da36e31305f1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.151281 May 23 16:17:21.232188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1802/3712] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:22.090298 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:22.090890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:22.098041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:22.098488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:22.099259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:22.099694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:22.227430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:22.235618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2cd68a36-9cee-4759-981f-ec77499ff0b0 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.148939 May 23 16:17:22.236192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1911/3713] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:22.248999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:22.249618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:22.256074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:22.256525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:22.257288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:22.257726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:22.384931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:22.392822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-01356f37-b5fe-4264-9abc-8d6a11f4bdcf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.147673 May 23 16:17:22.399248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1803/3714] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:23.252443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:23.253111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:23.259724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:23.260172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:23.260924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:23.261390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:23.384708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:23.392582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a3acd8e-4ec1-4295-8f3d-98d74c12b9de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.143985 May 23 16:17:23.393156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1912/3715] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:23.410140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:23.410761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:23.413072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=52 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:23.417858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:23.418309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:23.419066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:23.419530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:23.547550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:23.555368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-951d8849-14c5-4605-98b9-578c6ee775dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.149649 May 23 16:17:23.555931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1804/3716] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:24.409480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:24.410153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:24.412470 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=52 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:24.417188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:24.417635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:24.418387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:24.418820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:24.545483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:24.553818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-58f26e70-f136-46ca-9be6-5a01c730b860 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.148128 May 23 16:17:24.554386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1913/3717] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:24.572708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:24.573326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:24.580130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:24.580580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:24.581339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:24.581767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:24.725653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:24.733535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-62d6c17e-e540-40b9-b532-fcfcf6205ee8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.164598 May 23 16:17:24.734086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1805/3718] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:25.570675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:25.571274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:25.577840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:25.578281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:25.579026 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:25.579486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:25.710171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:25.718119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-95516a3f-fe38-4ad5-ab71-f7702643c653 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151820 May 23 16:17:25.718685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1914/3719] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:25.751080 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:25.751809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:25.758503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:25.758952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:25.759740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:25.760174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:25.890581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:25.899356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4cdbbc2-39a1-417c-8ef8-d2830261b8e1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.152016 May 23 16:17:25.900538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1806/3720] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:26.734303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:26.734926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:26.741451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:26.741899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:26.742649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:26.743083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:26.866264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:26.874124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9a8f5d5d-efc4-4843-ab49-ca0edf0ba987 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.143679 May 23 16:17:26.874723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1915/3721] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:26.916085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:26.916703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:26.923167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:26.923615 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:26.924360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:26.924790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:27.048057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:27.055888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5eb27d79-b038-4005-bb92-37b64fbc2d5d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.143585 May 23 16:17:27.056447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1807/3722] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:27.891711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:27.892314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:27.899084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:27.899553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:27.900331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:27.900759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:28.072603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:28.073209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:28.080015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:28.080473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:28.081253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:28.081694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:28.334682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:28.338605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:28.342689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dfd17c86-689d-4822-98e5-75eabbdc7367 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.454767 May 23 16:17:28.343299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1916/3723] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:28.346575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-91f334cb-c3fe-4962-ba4a-e02b0a059cae tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.277879 May 23 16:17:28.347130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1808/3724] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:29.360466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:29.361079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:29.363826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:29.365068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:29.368188 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:29.368643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:29.369394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:29.369828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:29.371536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:29.371988 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:29.372745 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:29.373218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:29.550649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:29.558530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a1acb261-f183-4a16-bc50-a2e005cb65a9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.202127 May 23 16:17:29.559123 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1917/3725] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:29.563998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:29.571777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-952cfee4-b5d4-4183-b0ec-58ac05fe4ef7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211703 May 23 16:17:29.572332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1809/3726] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:30.575543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:30.576078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:30.582531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:30.582975 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:30.583752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:30.584187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:30.587793 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:30.588429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:30.594848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:30.595348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:30.596120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:30.596567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:30.775541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:30.784152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:30.784313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4a1ff8ce-bcbb-46bf-8d29-38022bf0a1df tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211785 May 23 16:17:30.784542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1918/3727] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:30.791319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-93406843-cd47-405b-a695-8f3b9b970b2a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.207384 May 23 16:17:30.791891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1810/3728] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:31.800969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:31.801582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:31.808646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:31.808855 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:31.809290 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:31.809406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:31.809828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:31.810259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:31.815821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:31.816271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:31.817030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:31.817465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:31.998884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:32.005492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:32.007728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4edf9cab-6011-4d48-80fa-740173b20eeb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210555 May 23 16:17:32.008311 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1919/3729] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:32.014053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4a7934d4-48eb-4f6a-a661-f19bdb3cb24a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209144 May 23 16:17:32.014802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1811/3730] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:33.025284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:33.025949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:33.030930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:33.031573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:33.032552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:33.033148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:33.033754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:33.034185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:33.037937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:33.038392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:33.039174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:33.039613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:33.230765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:33.234681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:33.236676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:33.240541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3112a7f2-7435-4fb9-aaa7-921553c42cfc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219395 May 23 16:17:33.241439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1920/3731] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:33.244065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-af78044c-ca08-4ddf-ad06-766751a72a4c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216846 May 23 16:17:33.244628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1812/3732] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:34.260101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:34.260730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:34.261607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:34.262191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:34.268380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:34.268839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:34.269612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:34.270174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:34.283548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:34.283548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:34.283548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:34.283548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:34.465949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:34.474055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f702fef1-dd6e-48fb-8ba0-beadb23f3baa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218224 May 23 16:17:34.474727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1921/3733] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:34.482235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:34.490456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b4a6a900-0282-4bfc-80e5-8351ccf3b126 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.232622 May 23 16:17:34.491115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1813/3734] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:35.491965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:35.492611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:35.499056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:35.499540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:35.500295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:35.500731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:35.507652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:35.508270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:35.514785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:35.515284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:35.516051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:35.516491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:35.691270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:35.695783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:35.699712 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3e205952-84c6-4297-a6d7-13449f02e18b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212183 May 23 16:17:35.700284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1922/3735] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:35.703564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6a37a554-12aa-4f16-97b6-6a807118442a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.200328 May 23 16:17:35.704110 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1814/3736] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:36.717158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:36.717704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:36.720202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:36.720823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:36.723858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:36.724308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:36.725584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:36.726023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:36.727651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:36.728106 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:36.728897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:36.729340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:36.915162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:36.919187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:36.923504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bfdc5314-85f5-431b-a9b5-a8093d32ace9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210507 May 23 16:17:36.924163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1923/3737] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:36.927133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2121b4e4-3016-419b-8e14-29fedf774067 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210642 May 23 16:17:36.928330 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1815/3738] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:37.941386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:37.942017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:37.945042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:37.945665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:37.949167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:37.949611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:37.950437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:37.950801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:37.951649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:37.952107 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:37.952867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:37.953305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:38.142765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:38.145852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:38.152236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-281fd62a-c8cf-4357-804a-13e1a79dffed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214775 May 23 16:17:38.152808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1924/3739] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:38.154120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84533701-1bbb-49c9-91a6-9ac8b341052d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213434 May 23 16:17:38.154755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1816/3740] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:39.169418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:39.170045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:39.170600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:39.171280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:39.176569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:39.177010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:39.177579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:39.177756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:39.178032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:39.178187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:39.178792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:39.179765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:39.368696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:39.372187 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:39.376841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f4b695c6-be29-4215-91b1-fd28a3f82025 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211456 May 23 16:17:39.377460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1925/3741] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:39.380513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fb0f5587-c705-441a-be91-e4d2707ec1f0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213658 May 23 16:17:39.381074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1817/3742] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:40.394268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:40.394878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:40.398990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:40.399625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:40.402288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:40.402732 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:40.403515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:40.403951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:40.406008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:40.406470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:40.407267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:40.407714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:40.424997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:40.594465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:40.598455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:40.602406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08207a48-47ff-4270-afa3-47a719f951e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211952 May 23 16:17:40.602966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1926/3743] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:40.606512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aab9b70c-9c63-4ae7-a63b-e06ce0f03ffc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212165 May 23 16:17:40.607047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1818/3744] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:41.619865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:41.620561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:41.627306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:41.627756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:41.628508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:41.628937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:41.630368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:41.631040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:41.637998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:41.638606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:41.639260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:41.639694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:41.824569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:41.827553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:41.832662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5f822a48-ed6f-4ade-8045-a49bb6d99028 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217058 May 23 16:17:41.833230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1927/3745] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:41.836473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-16bab8b7-90a9-4487-ba2d-39eb4d9e77d6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211071 May 23 16:17:41.837026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1819/3746] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:42.850007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:42.850671 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:42.853769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:42.854388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:42.857667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:42.858124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:42.858874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:42.859343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:42.860731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:42.861212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:42.861968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:42.862407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:43.049948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:43.056636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:43.058521 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-75082706-1749-46b6-aaec-657cf12d2b84 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212147 May 23 16:17:43.059210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1928/3747] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:43.064626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-115fb662-4b43-46c6-a4e2-ee3526632877 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215226 May 23 16:17:43.065186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1820/3748] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:44.075911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:44.076689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:44.081545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:44.082165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:44.083205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:44.083653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:44.084408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:44.084837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:44.088572 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:44.089010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:44.089768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:44.090206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:44.277870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:44.281941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:44.286790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3d55f630-e451-4b51-814a-fe33d7a07c32 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214694 May 23 16:17:44.287483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1929/3749] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:44.290252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3d7b489a-0f12-4056-9ea9-478f92e0c54c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212688 May 23 16:17:44.290833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1821/3750] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:45.304823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:45.305435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:45.307488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:45.308109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:45.312172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:45.312619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:45.313375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:45.313808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:45.314676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:45.315156 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:45.315918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:45.316355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:45.506456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:45.510919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:45.514978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c8321665-219b-4e65-8608-29e435b406f8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214510 May 23 16:17:45.515577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1930/3751] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:45.518842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c15bbe16-bcb8-43a1-8d8c-d45b0a4e25a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215684 May 23 16:17:45.519432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1822/3752] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:46.532552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:46.533160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:46.535813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:46.536445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:46.539703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:46.540145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:46.540901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:46.541361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:46.543006 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:46.543486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:46.544254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:46.544713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:46.730514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:46.736880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:46.738698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ffdd92fd-1502-4b2e-90b3-117b8a9fe705 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209985 May 23 16:17:46.739389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1931/3753] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:46.745414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d43baa1f-f4ac-4828-9d76-2cee87fde667 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213425 May 23 16:17:46.745972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1823/3754] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:47.756496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:47.757183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:47.762239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:47.762853 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:47.764062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:47.764520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:47.765275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:47.765835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:47.769449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:47.769915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:47.770666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:47.771195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:47.984400 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:47.988392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:47.996113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-52d37eb6-aac4-478e-8c2b-076bfa81b98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.243204 May 23 16:17:47.996412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-51457cf8-a5d2-45ac-81cb-effa020e70d1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.237917 May 23 16:17:47.996706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1932/3755] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:47.996991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1824/3756] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:49.012142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:49.012797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:49.013151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:49.013768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:49.019202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:49.019696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:49.019913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:49.020368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:49.020509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:49.020913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:49.021120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:49.021554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:49.218758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:49.222709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:49.227455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1622e2ca-12bd-4405-9d38-2ed56ac121f4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219119 May 23 16:17:49.228044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1933/3757] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:49.231360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-51d72746-f949-4b9f-832b-2349f24aa59d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.221915 May 23 16:17:49.231922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1825/3758] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:50.245048 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:50.245666 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:50.248787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:50.249414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:50.252301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:50.252751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:50.253542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:50.253980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:50.256077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:50.256746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:50.257546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:50.259457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:50.449664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:50.452710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:50.457641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-60fb8f38-6cfd-462c-8288-cbefc8e397b6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217115 May 23 16:17:50.458412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1934/3759] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:50.461274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-817d0f15-a3fa-4c7d-8eae-3233748069de tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215754 May 23 16:17:50.461822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1826/3760] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:51.475480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:51.476093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:51.477980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:51.478594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:51.482491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:51.482938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:51.483743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:51.484176 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:51.484882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:51.485379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:51.486083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:51.486516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:51.673001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:51.681448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b5b37cc-eb87-4b7c-b409-16fb0389ed17 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210127 May 23 16:17:51.682017 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1935/3761] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:51.684387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:51.692368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d1f35ba-9f6e-4021-b5a7-11ac5c3a19b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218263 May 23 16:17:51.692928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1827/3762] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:52.698962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:52.699583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:52.705983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:52.706429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:52.707208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:52.707649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:52.708569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:52.709186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:52.715606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:52.716057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:52.716823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:52.717259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:52.940042 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:52.940226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:52.940355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d6ff7f87-15b5-4407-9cf4-18fffc5270be tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.202249 May 23 16:17:52.940355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1828/3763] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:52.940558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-666ab7a6-8b72-4af9-9458-7b697965fd73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215459 May 23 16:17:52.940558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1936/3764] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:53.924323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:53.924948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:53.926768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:53.927251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:53.927410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:53.932083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:53.932540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:53.933301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:53.933735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:53.933869 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:53.934253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:53.935004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:53.935467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:53.953420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:54.125826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:54.128883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:54.134366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5f469866-2df8-4407-9dc1-97a8b4ee6c58 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213934 May 23 16:17:54.135130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1829/3765] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:54.137401 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b0788eb-6ebb-4a67-b323-cb17963c1ef4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214394 May 23 16:17:54.137961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1937/3766] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:55.151835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:55.152563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:55.154145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:55.154773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:55.157154 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:55.166902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:55.166902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:55.166902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:55.166902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:55.167564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:55.167564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:55.167564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:55.167564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:55.361856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:55.367119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:55.372518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35c7b38e-5aa6-411e-ab7b-2635248c3ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.225130 May 23 16:17:55.373390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1830/3767] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:55.375552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d5f2b4d5-9828-4fa8-85ab-01733cd3c8f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225341 May 23 16:17:55.376191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1938/3768] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:56.389811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:56.390435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:56.392421 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:56.393025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:56.397125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:56.397577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:56.398862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:56.399326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:56.399853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:56.400299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:56.401044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:56.401472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:56.590180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:56.593434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:56.598211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-06c8b0bc-b52c-497a-843c-c07796449baf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212275 May 23 16:17:56.598867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1831/3769] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:56.602177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-16931d9f-869d-4e95-adec-cf82b5742b3e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212844 May 23 16:17:56.602735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1939/3770] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:57.615376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:57.615952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:57.619360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:57.619979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:57.623148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:57.623608 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:57.624375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:57.624815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:57.626600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:57.627053 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:57.628022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:57.628510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:57.816943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:57.819941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:57.824906 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55f5f94b-b07d-4df2-98ae-46e68ef08d96 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213509 May 23 16:17:57.825585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1832/3771] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:57.827864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf041e1f-5e83-4ae4-878f-2e824df83233 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212899 May 23 16:17:57.828424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1940/3772] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:17:58.842685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:58.843356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:58.844309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:17:58.844922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:17:58.850027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:58.850481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:58.851269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:58.851397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:58.851710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:58.851819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:58.852456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:17:58.852889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:17:59.043117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:59.046632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:17:59.051329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-304fb657-c1cf-4608-8f8c-0f141f7582aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212486 May 23 16:17:59.051908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1833/3773] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:17:59.054809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b72cfbb-5735-4e6c-a78f-6edbe60421fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214254 May 23 16:17:59.055913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1941/3774] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:17:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:00.069108 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:00.069709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:00.071394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:00.072021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:00.076914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:00.077363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:00.078126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:00.078402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:00.078563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:00.078860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:00.079656 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:00.080098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:00.272233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:00.275616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:00.280555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e753b2f7-c23f-4cf6-a823-ec52b51177fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215327 May 23 16:18:00.281127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1834/3775] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:00.283717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-92555791-2d99-4f51-ab04-50448dcd9aa3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216603 May 23 16:18:00.284281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1942/3776] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:01.297685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:01.298304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:01.300260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:01.300874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:01.305047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:01.305505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:01.306274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:01.306716 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:01.307253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:01.307696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:01.308445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:01.308875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:01.310843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:01.494670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:01.504215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:01.504408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4151130-5976-40a3-aa7a-98491f197570 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208878 May 23 16:18:01.504408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1835/3777] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:01.510180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-69227553-437d-4227-8aec-f2f162131042 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213766 May 23 16:18:01.510746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1943/3778] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:02.520304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:02.522177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:02.528198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:02.528899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:02.530116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:02.530577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:02.531728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:02.531862 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:02.535035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:02.535532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:02.536292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:02.536729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:02.721198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:02.724879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:02.729863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d8e4e47e-c9a2-4ba1-ae3d-3d4421b1e992 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213469 May 23 16:18:02.730445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1836/3779] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:02.734511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fc6ed36c-d141-48f1-8132-5e9c45dc4865 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210949 May 23 16:18:02.735084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1944/3780] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:03.747819 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:03.748440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:03.750958 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:03.751649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:03.754955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:03.755437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:03.756208 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:03.756645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:03.757956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:03.758403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:03.759194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:03.759632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:04.187687 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:04.191786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:04.195575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a877c59b-79e5-4cc5-98dd-5650a9e3a90e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.451638 May 23 16:18:04.196200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1837/3781] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:04.199816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d8a39733-cc8a-42f7-b20c-e631d5f14f9e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.452603 May 23 16:18:04.200375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1945/3782] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:05.212141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:05.212767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:05.216295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:05.216951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:05.219596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:05.220056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:05.220828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:05.221331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:05.223331 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:05.223790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:05.224545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:05.224980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:05.421015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:05.425229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:05.429154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-59abcdde-9311-430a-9259-ddf8c27d0af7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220990 May 23 16:18:05.429730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1838/3783] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:05.433653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f689de56-89f7-45de-ad93-ad840860b28b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.221112 May 23 16:18:05.434221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1946/3784] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:06.446310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:06.446897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:06.450075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:06.450686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:06.453552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:06.454004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:06.454764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:06.455234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:06.457066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:06.457517 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:06.458261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:06.458695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:06.647984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:06.653014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:06.656870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-148180f8-73e1-4ada-a4e8-5fb360af6508 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210605 May 23 16:18:06.657454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1947/3785] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:06.660934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-80f0b282-c142-456e-b42d-eebc099e3ead tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218684 May 23 16:18:06.661492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1839/3786] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:07.673741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:07.674304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:07.678094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:07.678707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:07.680760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:07.681209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:07.681959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:07.682387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:07.684841 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:07.685292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:07.686053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:07.686490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:07.874319 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:07.878145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:07.882500 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3844686c-1716-4423-9e3e-30920569c199 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212680 May 23 16:18:07.883072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1948/3787] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:07.886124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-35cca60c-3c76-4453-b914-7086f0248573 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212071 May 23 16:18:07.886676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1840/3788] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:08.899028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:08.899667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:08.903140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:08.903766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:08.906129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:08.906573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:08.907349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:08.907778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:08.910181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:08.910633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:08.911425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:08.911867 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:09.100220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:09.104694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:09.108670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0aa49dfd-5de7-4a2d-a831-01c53cc0803e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213744 May 23 16:18:09.109239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1949/3789] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:09.112653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5887a8b-cfa2-421f-807c-a75590d80173 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213349 May 23 16:18:09.113211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1841/3790] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:10.125854 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:10.126410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:10.130265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:10.130897 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:10.133002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:10.133460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:10.134223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:10.135257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:10.145531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:10.145980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:10.146749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:10.147209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:10.325404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:10.328730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:10.333603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d9494389-3489-4c6d-8431-60457e808b9a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211685 May 23 16:18:10.334172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1950/3791] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:10.336581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d4a0e52c-8b2e-4325-a657-977cc32e8585 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210242 May 23 16:18:10.337141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1842/3792] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:11.350468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:11.351081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:11.352808 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:11.353422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:11.357875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:11.358335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:11.359257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:11.359707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:11.359858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:11.360207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:11.360956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:11.361384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:11.552087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:11.555915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:11.560731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ba02040a-17e7-4d02-a7f3-ca7a1d636f55 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214174 May 23 16:18:11.561504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1951/3793] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:11.564430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36d9cf24-a427-4c1f-b90e-2b26069da955 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215358 May 23 16:18:11.565071 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1843/3794] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:12.578175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:12.578782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:12.585693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:12.585893 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:12.586310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:12.586526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:12.586867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:12.587326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:12.593555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:12.593999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:12.594826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:12.595282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:12.779969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:12.784249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:12.787973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78c0dc74-ac4b-42d5-8c58-7af88ed7e98f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213548 May 23 16:18:12.788553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1952/3795] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:12.792597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7645dd7e-51dc-4d76-929c-335c955b95ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210390 May 23 16:18:12.793297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1844/3796] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:13.804796 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:13.805392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:13.809308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:13.809935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:13.811813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:13.812258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:13.813007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:13.813436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:13.816786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:13.818351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:13.818351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:13.818351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:14.005194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:14.009256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:14.013976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-38f66e16-301b-4e02-b837-0bc738c22112 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213022 May 23 16:18:14.014551 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1953/3797] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:14.017681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6bb89290-860b-485c-a794-8caa7956d6b4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212158 May 23 16:18:14.018239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1845/3798] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:15.030973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:15.031617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:15.035529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:15.036145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:15.038196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:15.038639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:15.039447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:15.039879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:15.042962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:15.043440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:15.044209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:15.044370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:15.044647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:15.232867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:15.240667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:15.243444 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0c732cdc-7c0f-409f-81df-04222d05476d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216284 May 23 16:18:15.244031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1954/3799] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:15.249559 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d57b3196-6e3c-438e-a49c-d6b0902963a6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217834 May 23 16:18:15.250119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1846/3800] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:16.260318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:16.261064 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:16.266019 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:16.266663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:16.267558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:16.268008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:16.268766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:16.269412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:16.272829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:16.275644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:16.275644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:16.275644 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:16.463930 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:16.467919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:16.472213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-91295305-d73a-47be-87c8-2674da40f567 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215647 May 23 16:18:16.472817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1955/3801] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:16.475797 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0ed5ff6e-bb2f-4d4c-a958-924356bcd0a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213580 May 23 16:18:16.476348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1847/3802] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:17.489907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:17.490429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:17.493362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:17.493986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:17.497539 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:17.497991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:17.498749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:17.499214 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:17.500908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:17.501365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:17.502127 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:17.502566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:17.689107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:17.692357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:17.697216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eb5ffb9c-6d91-4d56-aa7c-3bad54438de6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211479 May 23 16:18:17.697946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1956/3803] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:17.700950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7d265111-1776-47f6-8832-9e553fcde001 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211393 May 23 16:18:17.701502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1848/3804] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:18.019560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:18:18.019765 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:18:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:18:18.715380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:18.715933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:18.718072 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:18.718695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:18.722359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:18.722808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:18.723600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:18.724047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:18.725273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:18.725734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:18.726494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:18.726935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:18.919440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:18.926070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:18.927278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9e85534e-e580-4b0d-a7a4-45d99ee86269 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213532 May 23 16:18:18.927845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1849/3805] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:18.934533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f9663120-ccbf-4cd2-b2a4-0227f465a2d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.223731 May 23 16:18:18.935115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1957/3806] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:19.944882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:19.945501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:19.951609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:19.952229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:19.952699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:19.953149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:19.953907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:19.954339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:19.958514 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:19.958964 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:19.959780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:19.960199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:20.150334 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:20.156868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-553e52f4-a0df-4a10-9d8f-45618ce6b650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216002 May 23 16:18:20.158032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1850/3807] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:20.159831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:20.168254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9eea9412-0acf-4bb9-a13a-0f52d630c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.220967 May 23 16:18:20.168828 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1958/3808] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:21.174105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:21.174733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:21.181311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:21.181768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:21.182533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:21.182975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:21.184792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:21.185448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:21.191895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:21.192340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:21.193100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:21.193538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:21.379361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:21.382376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:21.382707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:21.387507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b54a0be0-44f5-4df0-8330-d27cbe6575d5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217270 May 23 16:18:21.388070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1851/3809] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:21.390408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8824c40b-cb94-4887-a665-4aa8e87c3f10 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209364 May 23 16:18:21.390982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1959/3810] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:22.404380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:22.405057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:22.411799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:22.412291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:22.413040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:22.413473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:22.423734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:22.424355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:22.430720 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:22.431203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:22.431973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:22.432413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:22.611777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:22.615477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:22.621492 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f892f8e5-0bb7-40f7-aa66-1e518d158b97 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.221269 May 23 16:18:22.622587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1852/3811] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:22.625865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c773987d-7c99-4484-b26c-6231695438a1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.205884 May 23 16:18:22.626618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1960/3812] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:23.638856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:23.639427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:23.644004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:23.644622 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:23.645895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:23.646343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:23.647117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:23.647554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:23.650909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:23.651385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:23.652142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:23.652578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:23.836671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:23.840582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:23.845212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-04a2f341-9ba4-4102-9f25-536cc07c258b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210838 May 23 16:18:23.845771 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1853/3813] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:23.849186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad7752fe-93f6-4f14-93b3-d23987b2778c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208971 May 23 16:18:23.849743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1961/3814] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:23.967567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:18:23.967751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:18:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:18:24.644323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:18:24.644830 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:18:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:18:24.861628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:24.862242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:24.864596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:24.865408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:24.866009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:24.869419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:24.869872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:24.870639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:24.871079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:24.872220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:24.873173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:24.874092 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:24.874533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:25.070467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:25.073041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:25.079394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-42d4a156-37b3-4282-824a-cd7fb89c44b7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.221517 May 23 16:18:25.080367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1854/3815] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:25.082630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-533995d7-54ce-446f-9077-a07a4d500c6a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.220921 May 23 16:18:25.083289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1962/3816] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:26.096987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:26.097635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:26.099324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:26.100118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:26.102428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:26.104228 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:26.104689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:26.105463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:26.105912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:26.107632 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:26.108076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:26.108830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:26.109262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:26.301116 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:26.304527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:26.309246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4d2ff093-4b5d-4f68-91f8-8d6153fb1f53 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216212 May 23 16:18:26.309892 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1855/3817] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:26.312965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d049d99b-e807-4298-b0b2-8e83babc9fff tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217570 May 23 16:18:26.313530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1963/3818] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:27.328701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:27.329262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:27.329857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:27.330467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:27.335760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:27.336212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:27.336753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:27.336972 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:27.337202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:27.337409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:27.337947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:27.338379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:27.530175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:27.533941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:27.540639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5ef4ff12-6ba2-45be-b57e-2d37dba49115 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216221 May 23 16:18:27.541535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1856/3819] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:27.542125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e8d763d9-d4f6-42ce-b12f-35b60377e39c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216352 May 23 16:18:27.542690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1964/3820] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:28.559843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:28.560896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:28.562950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:28.563601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:28.567370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:28.567863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:28.568637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:28.569077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:28.569857 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:28.570302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:28.571049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:28.571762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:28.758154 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:28.762100 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:28.766672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8989ad04-4e94-452d-8251-af8fa4b3c655 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210897 May 23 16:18:28.767291 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1857/3821] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:28.770544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7a9f3537-409a-4dfd-83d9-0d02aa7fb534 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211784 May 23 16:18:28.771125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1965/3822] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:29.785250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:29.785866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:29.787744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:29.788363 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:29.794130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:29.794770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:29.794914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:29.795250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:29.795798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:29.796006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:29.796443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:29.796554 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:29.994809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:29.998581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:30.003031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6d26c28f-f15e-4039-a984-22d8e941aacf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219266 May 23 16:18:30.005729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1966/3823] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:30.006598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-55f880a4-18ef-414c-8121-12c9f11763ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225935 May 23 16:18:30.007167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1858/3824] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:31.020230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:31.020838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:31.024525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:31.025126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:31.027908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:31.028392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:31.029162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:31.029599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:31.031966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:31.032415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:31.033163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:31.033593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:31.223158 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:31.226335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:31.231486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-80c6de53-6fa5-41c9-a688-ea6da778681c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215075 May 23 16:18:31.236255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1967/3825] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:31.236921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-db48d419-802b-44b2-a28d-8d1335d9419f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214724 May 23 16:18:31.236921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1859/3826] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:32.249713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:32.250412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:32.252565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:32.253179 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:32.256999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:32.257445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:32.258194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:32.258628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:32.259558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:32.260009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:32.260763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:32.261199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:32.449333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:32.453274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:32.458150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1ec9057a-44a6-43e2-8dc2-c5bf0664ade2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213116 May 23 16:18:32.458724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1968/3827] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:32.461283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-78a1df25-7750-4736-8397-01cda4a7d250 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213201 May 23 16:18:32.461839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1860/3828] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:33.475414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:33.476669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:33.483471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:33.483952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:33.484733 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:33.485259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:33.578904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:33.579545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:33.585477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:33.585916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:33.586668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:33.587115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:33.618898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:33.627435 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-92f08854-495a-41d5-a773-ad9044740ab8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.155911 May 23 16:18:33.630063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1969/3829] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:33.713694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:33.722259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf31e5f2-b3e5-4da6-869c-dd6be6f1bd4f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.248200 May 23 16:18:33.722820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1861/3830] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:34.645270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:34.645812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:34.652431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:34.652874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:34.653628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:34.654056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:34.739661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:34.740277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:34.746670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:34.747219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:34.747992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:34.748431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:34.783674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:34.787198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:34.792013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a8426441-f635-48c5-b8f2-153e8fd78d46 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.150712 May 23 16:18:34.792591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1970/3831] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:34.876513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:34.884426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3a44d6ab-a3fb-4948-ab27-fb0625e7eeaa tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.148654 May 23 16:18:34.884983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1862/3832] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:35.810010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:35.810566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:35.818101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:35.818554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:35.819338 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:35.819778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:35.901913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:35.902530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:35.909081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:35.909530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:35.910296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:35.910747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:35.946103 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:35.954300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-991f5b9b-8ca6-4df7-b34c-0d5c967e1fc3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.148284 May 23 16:18:35.954874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1971/3833] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:36.041026 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:36.048850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a3a431a0-82a7-4ad5-b55f-314f5ba30be2 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151068 May 23 16:18:36.049403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1863/3834] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:36.972110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:36.972709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:36.979263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:36.979714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:36.980476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:36.980914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:37.065743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:37.066438 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:37.073213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:37.073774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:37.074597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:37.075122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:37.106039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:37.114645 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e652018c-05f7-4da4-8042-68d7407c60e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.146900 May 23 16:18:37.115243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1972/3835] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:37.205272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:37.212970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ab3f81dc-a4d9-4d52-b10b-d43b2a309faf tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.151105 May 23 16:18:37.213530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1864/3836] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:38.131614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:38.132280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:38.138849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:38.139333 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:38.140641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:38.141094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:38.229460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:38.230086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:38.237866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:38.238395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:38.239295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:38.239820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:38.270001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:38.279276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1a636c13-2f90-4898-877f-18de436e76eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.151464 May 23 16:18:38.279858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1973/3837] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:38.365180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:38.372987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e85e860d-1cf9-4a3e-8952-33f6bf206346 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.147307 May 23 16:18:38.373540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1865/3838] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:38.808066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:18:38.808287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:18:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:18:39.297070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:39.297682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:39.304184 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:39.304628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:39.305373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:39.305808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:39.390727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:39.391367 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:39.398415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:39.398868 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:39.399943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:39.400385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:39.572645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:39.581421 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-951ceb43-87f6-4a68-8221-89f7e0fb9fa9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.195396 May 23 16:18:39.581995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1866/3839] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:39.615755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:39.623742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d229dc1b-d658-45bd-ae1d-0a330c81e2a8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.331093 May 23 16:18:39.624297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1974/3840] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:40.598276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:40.598826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:40.605324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:40.605782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:40.606546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:40.606984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:40.641652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:40.642280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:40.649062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:40.649790 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:40.650562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:40.650998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:40.793399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:40.802271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-83f272f6-8e8f-4768-860a-77747ca67197 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207933 May 23 16:18:40.802984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1867/3841] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:40.815613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:40.823700 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a160279-e0d9-4fbf-9f36-421fcd1de838 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.185802 May 23 16:18:40.825735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1975/3842] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:41.819648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:41.820273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:41.826783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:41.827763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:41.828526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:41.828957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:41.840874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:41.841467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:41.847722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:41.848274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:41.848795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:41.849549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:41.850009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:42.024131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:42.032360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f47c8762-35f3-4673-8b2e-3ed947f5c98a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216681 May 23 16:18:42.032931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1868/3843] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:42.037743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:42.045595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe9b9661-07e2-4f31-acc4-1991c1ffe5d8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208433 May 23 16:18:42.046142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1976/3844] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:43.049349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:43.050606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:43.057259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:43.057720 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:43.058478 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:43.058912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:43.061717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:43.062326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:43.068826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:43.069277 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:43.070033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:43.070461 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:43.254787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:43.264071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:43.264500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f30e9656-0a12-4d22-840d-afa452dbc3cd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218599 May 23 16:18:43.265878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1869/3845] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:43.270832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ff4e24f-1b0f-43e3-81fb-b51c9f319ad6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212871 May 23 16:18:43.271432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1977/3846] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:44.282820 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:44.283467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:44.290073 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:44.290530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:44.291325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:44.291768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:44.292087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:44.292715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:44.299270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:44.299722 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:44.300482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:44.300920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:44.477092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:44.485159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3cf024c4-b862-4425-af78-127609df62ab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.206100 May 23 16:18:44.485735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1870/3847] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:44.487496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:44.495374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f32c77b8-9a77-4246-909f-23ebcb471fc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.207269 May 23 16:18:44.495918 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1978/3848] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:45.502486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:45.503117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:45.509816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:45.510270 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:45.511028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:45.511495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:45.512533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:45.513146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:45.519597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:45.520052 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:45.520814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:45.521251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:45.702543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:45.706410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:45.710657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-75748709-63b6-4cb1-900a-7843e58aca63 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212576 May 23 16:18:45.711244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1871/3849] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:45.714186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b3350f66-d28c-447d-af02-d9134c994644 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.205454 May 23 16:18:45.714731 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1979/3850] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:46.728563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:46.729117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:46.731624 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:46.732238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:46.735643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:46.736104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:46.736870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:46.737331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:46.738494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:46.738934 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:46.739847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:46.740283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:46.752707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:18:46.752987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:18:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 (ip 10.222.0.32) !!! May 23 16:18:47.168588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:47.176205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:47.185094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89b235c6-ee5c-4288-bec0-14b2e4993553 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.452559 May 23 16:18:47.185094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1872/3851] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:47.185332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-298fa869-3c79-4127-8b3b-965297c580dc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.456589 May 23 16:18:47.185332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1980/3852] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:48.196426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:48.197124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:48.202229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:48.203416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:48.204376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:48.204838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:48.205612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:48.206056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:48.213013 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:48.213591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:48.214473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:48.215010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:48.408343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:48.412343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:48.416405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a0663098-fde8-4e3c-a80e-93e358049bc8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225885 May 23 16:18:48.416971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1873/3853] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:48.420190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-786450c1-3553-488e-b409-21c094aaf4c0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.221861 May 23 16:18:48.420738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1981/3854] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:49.434249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:49.434817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:49.438164 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:49.438806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:49.441490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:49.442186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:49.442688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:49.443143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:49.445408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:49.445859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:49.446610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:49.447079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:49.668824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:49.672055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:49.678482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73512acd-db9e-4254-952d-3684a78f5a02 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.246895 May 23 16:18:49.678482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1874/3855] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:49.681116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a003d94e-1e73-4474-8033-1100ac424c7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.245925 May 23 16:18:49.681116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1982/3856] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:50.693626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:50.694225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:50.698251 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:50.698875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:50.700998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:50.701444 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:50.702732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:50.703191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:50.705548 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:50.705998 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:50.706758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:50.707228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:50.886000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:50.894568 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e551d9a4-9c20-4b4d-b39c-de563fdcef8f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.204964 May 23 16:18:50.895929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1875/3857] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:50.897111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:50.906181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c780acd-a4d4-4413-ad36-14f4015ff2b9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211898 May 23 16:18:50.906772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1983/3858] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:51.911527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:51.912138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:51.919143 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:51.919597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:51.920360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:51.920791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:51.923714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:51.924334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:51.931124 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:51.931589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:51.932368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:51.932781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:52.114213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:52.117283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:52.122621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8ad33703-f9dc-45f6-af70-3100ec6a5495 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214932 May 23 16:18:52.123202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1876/3859] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:52.125112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-51d958dd-4ec6-4be2-9632-e5324ccf43da tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.205767 May 23 16:18:52.125669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1984/3860] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:53.139445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:53.140089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:53.141475 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:53.142223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:53.146624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:53.147084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:53.147854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:53.148323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:53.148553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:53.149003 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:53.150020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:53.150456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:53.346903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:53.350133 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:53.355147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9b4b77b3-5922-4d69-9ee1-7dcdafb8dd93 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219509 May 23 16:18:53.361235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1877/3861] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:53.361380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bcffbbf-abaf-4a45-8145-022f7c33f378 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220331 May 23 16:18:53.361380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1985/3862] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:54.372335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:54.372999 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:54.376860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:54.377481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:54.380317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:54.380848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:54.381520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:54.381955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:54.383996 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:54.384451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:54.385210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:54.385646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:54.576234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:54.578739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:54.584430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-615dca25-6e2e-470a-94cb-62eb25b4ab26 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212771 May 23 16:18:54.585082 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1986/3863] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:54.586580 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4d0e6bc-140e-41aa-a36a-650ac3e95e7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218291 May 23 16:18:54.587148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1878/3864] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:55.600933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:55.601480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:55.603008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:55.603666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:55.605673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:55.607955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:55.608406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:55.609208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:55.609646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:55.610312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:55.610773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:55.611565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:55.612012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:55.627909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:55.802254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:55.805346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:55.810670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2bcece24-af70-44ab-9f63-bb580d18408a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213762 May 23 16:18:55.811252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1987/3865] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:55.813933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7db2d6e7-a330-41dd-be48-2fe27f9c09f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214661 May 23 16:18:55.814483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1879/3866] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:56.829127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:56.829742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:56.831070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:56.831741 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:56.832085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:56.837253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:56.843015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:56.843730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:56.843868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:56.843974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:56.844085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:56.844192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:56.844192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:57.060066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:57.068520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6b3832e1-fcf4-4d80-b8f1-433dd838a4a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.243752 May 23 16:18:57.069098 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1988/3867] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:57.069570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:57.078268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cf5c7e26-9649-4d66-9e4e-35d5be58cf37 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.250861 May 23 16:18:57.078967 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1880/3868] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:58.086379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:58.086982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:58.094957 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:58.095931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:58.095931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:58.096132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:58.096261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:58.096706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:58.103805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:58.104263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:58.105034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:58.105475 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:58.290439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:58.292747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:58.298627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ae8e68ec-dbd5-49bc-a58a-15337f1f8e5b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216148 May 23 16:18:58.299228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1989/3869] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:18:58.300729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f1cf4f8f-70b1-48ee-8a94-c89d1352396b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209704 May 23 16:18:58.301456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1881/3870] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:59.316269 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:59.316878 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:59.317828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:18:59.318456 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:18:59.323320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:59.323770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:59.324536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:59.324829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:59.324969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:59.325285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:59.326050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:18:59.326490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:18:59.519744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:59.522831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:18:59.528038 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-44b73ecf-b485-40d6-ad75-d4b92d5dfdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214022 May 23 16:18:59.528739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1882/3871] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:18:59.531628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b7d9c24f-92d5-4ef7-85da-d061ee87c03f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219761 May 23 16:18:59.532189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1990/3872] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:18:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:00.545346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:00.545884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:00.548744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:00.549386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:00.552727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:00.553175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:00.553933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:00.554366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:00.555708 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:00.556151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:00.556904 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:00.557334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:00.741634 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:00.750114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-90c21a5c-d3dd-418d-a1c1-c92c9995f926 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208708 May 23 16:19:00.750700 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1883/3873] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:00.753037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:00.761055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e31db5d6-2f01-4876-b641-5ea481d21981 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216589 May 23 16:19:00.761614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1991/3874] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:01.766115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:01.766724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:01.773204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:01.773628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:01.774385 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:01.774815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:01.778594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:01.779247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:01.786139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:01.786590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:01.787388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:01.787830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:01.971628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:01.974780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:01.979588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4c171f3c-29b1-47d9-b231-451fec446a5f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217238 May 23 16:19:01.980441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1884/3875] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:01.982566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cab82224-095a-494b-b430-1f84adad9099 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208021 May 23 16:19:01.983141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1992/3876] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:02.996341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:02.996960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:02.999616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:03.000255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:03.004012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:03.004464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:03.005234 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:03.005674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:03.006553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:03.007015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:03.007817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:03.008259 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:03.009556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:03.198172 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:03.202343 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:03.206989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a5f21541-f1bf-42cc-9f21-181d73c95227 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214611 May 23 16:19:03.207597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1885/3877] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:03.210822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-28957ff8-6c3c-4551-9ac5-92e1b9119b77 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215513 May 23 16:19:03.211545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1993/3878] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:04.224852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:04.225397 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:04.227472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:04.228090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:04.231982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:04.232484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:04.233292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:04.233750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:04.234555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:04.235007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:04.235812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:04.236273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:04.426348 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:04.430010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:04.434379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-68194561-8aab-4b5a-91ae-c273e7e2414d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213412 May 23 16:19:04.434946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1886/3879] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:04.437941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-73cbf093-3f18-4470-b230-97ebcbaafac4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214183 May 23 16:19:04.438749 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1994/3880] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:05.452423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:05.453012 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:05.455061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:05.455698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:05.459527 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:05.459982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:05.460755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:05.461192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:05.462059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:05.462505 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:05.463282 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:05.463716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:05.652839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:05.656413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:05.661408 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-15118a0b-c28e-4c4d-a43f-4d166536bd04 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213499 May 23 16:19:05.662079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1887/3881] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:05.664328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9478b87d-b4bc-496c-885a-202175bf268b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213062 May 23 16:19:05.664881 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1995/3882] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:06.678310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:06.678854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:06.680868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:06.681482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:06.685382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:06.685838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:06.686603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:06.687039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:06.687538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:06.687987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:06.688737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:06.689172 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:06.873279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:06.880761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:06.881911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3d1be1a9-3c37-40e8-8b92-688d165428b1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207613 May 23 16:19:06.882482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1888/3883] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:06.888683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-304e0b3f-8e5d-44e0-9c5a-126485ed8bf3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211573 May 23 16:19:06.889237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1996/3884] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:07.899915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:07.900790 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:07.906775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:07.907414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:07.908124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:07.908579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:07.909346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:07.909792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:07.913858 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:07.914313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:07.915065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:07.915522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:08.102236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:08.106240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:08.110521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e28e0e5d-dff0-465a-bcf9-19cf63f73dfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214461 May 23 16:19:08.111207 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1889/3885] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:08.114073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-425daa9d-a86f-4324-bd3c-fd7abec5b4e4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211682 May 23 16:19:08.114623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1997/3886] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:09.128230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:09.128828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:09.130954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:09.131609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:09.135409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:09.135864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:09.136625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:09.137064 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:09.137992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:09.138453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:09.139262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:09.139698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:09.329267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:09.333101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:09.337235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a598739e-0d67-4db4-85ee-28899d5e0673 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213419 May 23 16:19:09.337811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1890/3887] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:09.341016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0d7c94c1-20f5-4a6f-ac75-1c2cc85a2d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213856 May 23 16:19:09.341573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1998/3888] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:10.353680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:10.354292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:10.357519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:10.358174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:10.360945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:10.361390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:10.362145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:10.362587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:10.364536 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:10.364977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:10.365724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:10.366159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:10.550544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:10.556851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:10.558842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a2b74959-8b29-40e8-b92b-d50b9d0beaef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208957 May 23 16:19:10.559465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1891/3889] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:10.566047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-91ad9743-4378-494c-93e4-fbc37897d123 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212315 May 23 16:19:10.566615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 1999/3890] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:11.576275 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:11.576903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:11.582887 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:11.583526 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:11.583654 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:11.584054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:11.584827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:11.585267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:11.589902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:11.590347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:11.591117 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:11.591594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:11.771359 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:11.779636 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-64bababa-b019-4f35-80c1-9c6276ac5e09 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207205 May 23 16:19:11.780206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1892/3891] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:11.782741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:11.790689 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eabd50da-2b30-4c8a-946e-8465043a03c5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211522 May 23 16:19:11.791256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2000/3892] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:12.802141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:12.802762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:12.808142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:12.808842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:12.810858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:12.811668 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:12.813094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:12.813599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:12.819812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:12.820275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:12.821170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:12.821618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:13.003623 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:13.006743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:13.011538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b54233d1-5864-4430-afe0-4c7eeabe8806 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213211 May 23 16:19:13.012305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1893/3893] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:13.014721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3b754c44-780d-4815-9ecb-ee5667f2891d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210529 May 23 16:19:13.015314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2001/3894] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:14.028593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:14.029204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:14.031544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:14.032627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:14.036355 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:14.049992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:14.049992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:14.049992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:14.050370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:14.050370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:14.050370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:14.050370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:14.232909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:14.240793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:14.243931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-32801c40-872d-47ec-8f16-fc2fc3e7219f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219233 May 23 16:19:14.244502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1894/3895] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:14.249044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-eeb64cd8-210f-4175-975c-f5df0335c6af tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.221234 May 23 16:19:14.249760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2002/3896] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:15.260876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:15.261493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:15.265982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:15.267278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:15.268871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:15.269323 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:15.270094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:15.270534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:15.273909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:15.274386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:15.275129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:15.275813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:15.459730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:15.463604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:15.467753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2c8c24b0-e33c-46eb-a6ec-eaff6238e29c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210733 May 23 16:19:15.468324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1895/3897] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:15.472119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b9c72c1d-b707-4294-9daf-2aad069bf303 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209689 May 23 16:19:15.472823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2003/3898] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:16.485287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:16.485912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:16.492597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:16.493053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:16.493816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:16.493952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:16.494389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:16.494513 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:16.501149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:16.501603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:16.502403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:16.502876 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:16.507931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:16.684509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:16.687768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:16.693035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd566c37-d33d-4010-828b-38e6e37cbf44 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211598 May 23 16:19:16.693691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1896/3899] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:16.696349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ee4eadd7-336c-439c-a51e-9632389afe67 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.206743 May 23 16:19:16.696905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2004/3900] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:17.710420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:17.711039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:17.713417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:17.714040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:17.717848 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:17.718310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:17.719133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:17.720282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:17.721089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:17.721549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:17.722308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:17.722748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:17.910004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:17.913785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:17.918020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e35fc372-3c2e-40d3-998e-c8beb29d1ff4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211488 May 23 16:19:17.918597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1897/3901] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:17.921676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5fadcbf2-60b7-4a55-b117-bceb1d835716 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211999 May 23 16:19:17.922235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2005/3902] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:18.935322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:18.935933 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:18.938063 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:18.938664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:18.943222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:18.943670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:18.944436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:18.944866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:18.945983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:18.946440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:18.947224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:18.947665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:19.379950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:19.386478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:19.387982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-893d2d8d-5408-4db4-a8ea-864cb083f1a5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.456589 May 23 16:19:19.388552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1898/3903] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:19.395518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c2e657da-9fba-4a32-9722-16f570cb97f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.461183 May 23 16:19:19.396213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2006/3904] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:20.405870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:20.406532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:20.412458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:20.413071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:20.413689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:20.414152 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:20.414932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:20.415399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:20.419657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:20.420132 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:20.420939 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:20.421372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:20.614420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:20.617742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:20.622398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8f4d4335-e614-40cc-8b96-6d0e8addf745 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213700 May 23 16:19:20.622969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2007/3905] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:20.626139 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ff3321d5-56e1-4b7a-8c1c-ac0001923612 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.224116 May 23 16:19:20.626697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1899/3906] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:21.640025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:21.640593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:21.642341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:21.642962 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:21.647325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:21.647782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:21.648554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:21.648991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:21.649482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:21.649939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:21.650975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:21.651440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:21.840778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:21.843759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:21.849486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6474f4b3-f685-4540-83d9-fccebc17113c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211152 May 23 16:19:21.850053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1900/3907] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:21.852206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e4ddf5fb-853a-4308-8b31-2ac9e97d9bfb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216325 May 23 16:19:21.852760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2008/3908] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:22.867202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:22.867936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:22.869512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:22.870125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:22.875027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:22.876379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:22.877125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:22.877382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:22.877573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:22.878014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:22.878328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:22.878761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:23.077310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:23.080142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:23.083469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:23.085441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fe85c314-347b-413f-9937-c5ab456cf2ad tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219659 May 23 16:19:23.086007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2009/3909] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:23.088232 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f26d9914-c792-4879-8663-74976a95a7ca tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225254 May 23 16:19:23.088792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1901/3910] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:24.102442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:24.103059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:24.107417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:24.108068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:24.109524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:24.109973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:24.110727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:24.111181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:24.115000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:24.115476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:24.116240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:24.116675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:24.298375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:24.301730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:24.306379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ac583bd5-f71b-4c66-96f7-9a3741864589 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.207778 May 23 16:19:24.306991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2010/3911] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:24.309699 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-78e985e1-9d05-413f-b6f4-5e29af42e93c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.206056 May 23 16:19:24.310260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1902/3912] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:25.323890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:25.324509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:25.327303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:25.327927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:25.332152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:25.332830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:25.333763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:25.334216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:25.334880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:25.335368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:25.336137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:25.336695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:25.535489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:25.539946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:25.543730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b93afb4a-0d2f-49ec-8b5c-99a2c16e00eb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.223865 May 23 16:19:25.544317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2011/3913] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:25.549341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb0e0777-0230-4b20-963d-00c8cc70a5a6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.226607 May 23 16:19:25.550081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1903/3914] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:26.561036 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:26.561660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:26.567242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:26.572373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:26.572508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:26.572508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:26.572508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:26.572508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:26.573048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:26.574826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:26.575322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:26.576079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:26.576553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:26.766816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:26.771032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:26.774874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-990e08ea-064e-41fd-aaa9-0a73f336c996 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217714 May 23 16:19:26.775426 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2012/3915] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:26.779342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c3ae0f9-f8e0-42c1-855e-9c8eeeb8fef7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216145 May 23 16:19:26.780055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1904/3916] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:27.793118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:27.793725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:27.796034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:27.797432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:27.798632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:27.801326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:27.801770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:27.802577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:27.803006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:27.805046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:27.807121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:27.807121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:27.807121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:27.995366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:27.999206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:28.003293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-01fe2d63-e785-40cf-b726-64883e3d8a2b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213964 May 23 16:19:28.003860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2013/3917] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:28.007282 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-39cb08ab-05df-455f-9392-175c83f977e9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213653 May 23 16:19:28.007837 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1905/3918] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:29.020824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:29.021462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:29.024895 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:29.025501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:29.028263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:29.028717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:29.029484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:29.029922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:29.031641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:29.033002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:29.033753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:29.034185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:29.230710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:29.234913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:29.239730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-53c5c450-0cb6-4f8a-9ade-3b322df36c9b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.223240 May 23 16:19:29.240318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2014/3919] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:29.244124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72c37cae-99d0-44ed-a016-bce7553d53f9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.224186 May 23 16:19:29.244685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1906/3920] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:30.256979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:30.257591 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:30.261415 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:30.262034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:30.264081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:30.264527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:30.265285 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:30.265719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:30.269204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:30.269655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:30.270460 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:30.270887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:30.450741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:30.458745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a19c9da2-801e-4a5f-92ee-b6ef84d9e4e0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.205553 May 23 16:19:30.459353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2015/3921] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:30.461734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:30.469734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-774f73d4-2183-4879-bce4-e67fc6106662 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212186 May 23 16:19:30.470300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1907/3922] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:31.476792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:31.477414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:31.483970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:31.484430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:31.485250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:31.486046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:31.486464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:31.486690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:31.493167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:31.493614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:31.494375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:31.494903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:31.684199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:31.690968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:31.692433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-579f9db8-2179-41f4-9af0-9fa0dff227b5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219481 May 23 16:19:31.693210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2016/3923] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:31.699194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b3555b7-723e-49e1-b00d-c057d667e0ba tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217461 May 23 16:19:31.699752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1908/3924] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:32.709754 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:32.710324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:32.715744 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:32.716384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:32.716678 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:32.717126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:32.717883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:32.718341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:32.722725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:32.723211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:32.723978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:32.724416 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:32.910484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:32.913426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:32.918427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-aca7d19b-ab72-47db-858d-f825e75ee1df tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212825 May 23 16:19:32.919205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2017/3925] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:32.921361 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f34e7947-e5c6-4e1b-9f28-9e4516f873b4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209358 May 23 16:19:32.921923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1909/3926] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:33.936157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:33.936781 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:33.938472 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:33.939079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:33.943799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:33.944256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:33.945021 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:33.945462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:33.945937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:33.946389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:33.947251 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:33.947602 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:34.128312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:34.136494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72fce982-7e0e-476a-8288-4573731b9a2d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.201770 May 23 16:19:34.137093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1910/3927] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:34.141851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:34.149611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1d9cf370-b634-404f-bf2d-7e5d44cffa7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217565 May 23 16:19:34.150162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2018/3928] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:35.152851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:35.153498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:35.160341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:35.160785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:35.161535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:35.161996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:35.167001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:35.167653 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:35.174655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:35.176586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:35.176586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:35.176586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:35.365887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:35.372984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:35.373864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a15fd2bd-6d8d-45d1-8fff-bdf37117870d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.225198 May 23 16:19:35.374447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1911/3929] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:35.381197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-52605196-b672-49f8-96b1-fd12b9761dc6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218042 May 23 16:19:35.381769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2019/3930] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:36.390641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:36.405041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:36.405209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:36.405655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:36.405809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:36.405925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:36.405925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:36.405925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:36.406216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:36.406216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:36.406216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:36.406216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:36.595395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:36.598602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:36.602167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:36.603406 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3175b953-67e6-4cc3-9f09-55a83573c2de tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216544 May 23 16:19:36.603977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1912/3931] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:36 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:36.607069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-211aa92d-9564-477f-b11a-2b09d055d1f2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213399 May 23 16:19:36.607646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2020/3932] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:36 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:37.620929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:37.621481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:37.628588 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:37.629041 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:37.629684 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:37.630123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:37.630237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:37.630353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:37.637099 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:37.637545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:37.638294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:37.638726 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:37.817425 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:37.825764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1e0c3fc7-bd46-476b-9273-b57e42b45c4b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208599 May 23 16:19:37.826184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1913/3933] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:37.828182 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:37.836163 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c501b2f8-0919-4764-9ae6-454e032decd1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210782 May 23 16:19:37.836711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2021/3934] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:38.842353 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:38.842970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:38.850093 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:38.850544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:38.851329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:38.851766 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:38.852808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:38.853415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:38.859952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:38.860399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:38.861294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:38.861729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:39.042490 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:39.045730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:39.050671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5b456ff2-de42-48b8-a617-e15b2ff09f22 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212144 May 23 16:19:39.051362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1914/3935] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:39.053549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-06f01d5f-761b-4407-b2a3-32df27a31b0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.204521 May 23 16:19:39.054140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2022/3936] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:40.068410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:40.069009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:40.070585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:40.071240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:40.075908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:40.076366 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:40.077137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:40.077250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:40.077577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:40.077694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:40.078351 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:40.078778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:40.264364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:40.270962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:40.272339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-de5e5ea6-2b8d-40cd-9942-7bc437f256c7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.208365 May 23 16:19:40.272918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1915/3937] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:40.279144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0aba8d5e-2b23-4779-80a7-6450ac5f535e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212325 May 23 16:19:40.279702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2023/3938] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:41.289407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:41.291646 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:41.296102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:41.296677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:41.296824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:41.297133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:41.297903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:41.298345 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:41.303935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:41.304387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:41.305147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:41.305586 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:41.493493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:41.495823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:41.501507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7aa7484a-9b19-4874-95f6-6ae09f816d92 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215921 May 23 16:19:41.502081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1916/3939] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:41.504562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a14dc2d1-c7ed-4aea-a9a8-90aeb113e65c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211669 May 23 16:19:41.505118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2024/3940] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:42.519290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:42.520390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:42.521327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:42.522037 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:42.527192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:42.531764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:42.531764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:42.532127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:42.532248 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:42.532367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:42.532367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:42.532367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:42.722521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:42.726046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:42.730727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2f10fb97-7845-4a71-b964-71b11fd7790b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215368 May 23 16:19:42.731329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1917/3941] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:42 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:42.737889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fbdafb80-06df-4f18-be3f-eb60c008c177 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.220610 May 23 16:19:42.738935 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2025/3942] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:42 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:43.748091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:43.748708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:43.755258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:43.755725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:43.755856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:43.755976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:43.756581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:43.757016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:43.762186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:43.762631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:43.763457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:43.763837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:43.775746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:43.951787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:43.955768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:43.960318 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f394b76b-2bb0-419a-808b-4fdb8a81a5e8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216072 May 23 16:19:43.960882 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1918/3943] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:43.964344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a4f3c08d-ff70-42b0-b679-91cb4affe20d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212842 May 23 16:19:43.964900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2026/3944] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:44.978129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:44.978742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:44.981422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:44.982047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:44.985305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:44.985752 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:44.986510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:44.986946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:44.989115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:44.989567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:44.990332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:44.990772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:45.180983 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:45.184782 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:45.189396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-61d40c43-035c-4a2e-a0c6-12c953563e19 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215079 May 23 16:19:45.189966 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1919/3945] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:45.193031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6312110f-76d3-41b9-9c7b-b89e5a553bed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215049 May 23 16:19:45.193600 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2027/3946] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:46.205874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:46.206491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:46.209681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:46.210288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:46.213044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:46.213503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:46.214265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:46.214704 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:46.216766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:46.217215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:46.217963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:46.218397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:46.405402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:46.408365 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:46.414047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-471b1462-c070-474f-9570-7984f619aa1d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212090 May 23 16:19:46.417222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-030a215b-c876-425b-9e5a-b6806b28f7db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211411 May 23 16:19:46.417795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2028/3947] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:46.418261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1920/3948] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:47.430190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:47.430836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:47.435005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:47.435659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:47.437491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:47.437976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:47.438747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:47.439218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:47.442676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:47.443176 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:47.443946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:47.444389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:47.635241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:47.638173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:47.643312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9ebda4d8-f3a2-45c1-8c81-71757402e609 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217208 May 23 16:19:47.643879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2029/3949] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:47 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:47.646493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2dad31c8-fa3d-4ed9-a054-2b0dea53f93b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215366 May 23 16:19:47.647050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1921/3950] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:47 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:48.660630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:48.661209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:48.663315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:48.663932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:48.667718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:48.668169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:48.668936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:48.669372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:48.670328 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:48.670783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:48.671567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:48.672000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:48.861097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:48.865856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:48.870531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-06274646-27ff-4bb8-92e3-c9370f086262 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214366 May 23 16:19:48.871157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2030/3951] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:48.873777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-905dda47-5339-4e71-b7f9-37137afdf355 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214734 May 23 16:19:48.874583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1922/3952] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:49.888152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:49.888778 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:49.891874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:49.892509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:49.895572 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:49.896031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:49.896805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:49.897240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:49.898989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:49.899469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:49.909239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:49.909239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:50.103711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:50.106751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:50.111769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-295da7b7-40b9-42b7-99f1-96dedcbddb9f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.223791 May 23 16:19:50.112332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1923/3953] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:50.115654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c94d4525-a05c-47cf-b3d9-475293f001f1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.231551 May 23 16:19:50.116304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2031/3954] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:51.130011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:51.130614 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:51.132402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:51.133076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:51.137212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:51.137665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:51.138427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:51.138864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:51.139534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:51.139982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:51.140740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:51.141185 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:51.332108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:51.336059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:51.340054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c969bb22-29a9-4673-8774-d7dfe7448efb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211928 May 23 16:19:51.340621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2032/3955] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:51.344147 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f628a074-52da-4cb8-9ccb-46b7d7bbba81 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218623 May 23 16:19:51.344843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1924/3956] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:52.356961 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:52.357602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:52.360079 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:52.360702 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:52.364932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:52.365379 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:52.366134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:52.366564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:52.367711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:52.368171 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:52.368932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:52.369372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:52.559014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:52.561998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:52.567597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-36ca1d70-98bb-4db2-b0da-e2a852e57b7b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214393 May 23 16:19:52.568174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2033/3957] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:52 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:52.570974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c4ec4d05-ce74-4c00-a8fa-fe34b178fad2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214607 May 23 16:19:52.571563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1925/3958] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:52 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:53.584601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:53.585155 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:53.587708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:53.588365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:53.591592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:53.592042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:53.592797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:53.593230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:53.594786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:53.595283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:53.596051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:53.596491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:53.787062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:53.790952 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:53.795424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-26b677b2-e3f7-46d1-b7f3-0781b676ace6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214755 May 23 16:19:53.796122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2034/3959] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:53.799306 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3d6d4e13-af9b-4a10-b542-6f9e7e8e7863 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215311 May 23 16:19:53.799885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1926/3960] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:54.811992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:54.812602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:54.817036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:54.817653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:54.819585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:54.820031 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:54.820791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:54.821220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:54.823998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:54.824449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:54.825209 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:54.825653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:55.017263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:55.020783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:55.025180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0753e76d-f91d-4896-b66a-4af626ee6924 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217023 May 23 16:19:55.025747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2035/3961] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:55.029025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-56f4ce6c-f510-4f33-9c01-c7c85087af0a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216287 May 23 16:19:55.029582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1927/3962] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:56.042417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:56.043044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:56.045394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:56.046015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:56.049609 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:56.050062 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:56.050894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:56.051361 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:56.052357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:56.052803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:56.053658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:56.053979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:56.244382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:56.247288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:56.252797 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-74bf826c-09e3-4f82-ad11-6355f125d564 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214344 May 23 16:19:56.253455 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2036/3963] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:56.255674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-cdc36627-14d2-4b50-8436-2742d85d3868 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214041 May 23 16:19:56.256258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1928/3964] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:57.269844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:57.270457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:57.272439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:57.273061 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:57.275396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:57.278120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:57.278601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:57.279391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:57.279979 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:57.280391 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:57.280845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:57.281612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:57.282048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:57.297916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:57.723789 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:57.727783 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:57.731751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de3f9b1c-aae4-420e-877b-2855cc86dad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.465914 May 23 16:19:57.732327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2037/3965] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:57.735569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2a33e8af-3899-4a6c-ab31-b4c6cc270629 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.466885 May 23 16:19:57.736121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1929/3966] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:58.749200 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:58.749862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:58.752059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:58.752472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:58.752679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:58.757941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:58.758385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:58.766014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:58.766014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:58.766266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:58.766266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:58.766266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:58.766266 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:58.952040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:58.955896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:19:58.960082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-34a885d3-5c6a-4ac1-98a8-cdfca74e7ae3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211875 May 23 16:19:58.960666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1930/3967] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:58 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:19:58.963822 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4ef7a73a-63af-4ae4-8b6f-6467124480c5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218459 May 23 16:19:58.964374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2038/3968] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:58 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:19:59.976910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:59.977517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:59.984118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:19:59.984641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:59.984755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:19:59.984875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:59.985522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:59.985960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:59.991481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:59.991941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:19:59.992746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:19:59.993186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:00.173948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:00.177054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:00.182159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-aed68604-7a48-462b-ab61-4c3e7986f3fb tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209149 May 23 16:20:00.182825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1931/3969] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:00.185040 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e1bc88ea-e570-4569-aaf8-0a9a30eb17fc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.204749 May 23 16:20:00.185594 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2039/3970] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:19:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:01.200087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:01.200641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:01.201471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:01.202104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:01.207354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:01.207805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:01.208511 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:01.208653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:01.208973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:01.209229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:01.209735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:01.210174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:01.397761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:01.401047 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:01.406136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-40452a45-e520-454e-b565-20101d0bd499 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209977 May 23 16:20:01.406885 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1932/3971] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:01 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:01.410481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-67eb460b-462c-4692-b376-c9faa565c3dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212225 May 23 16:20:01.411166 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2040/3972] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:01 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:02.424045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:02.424545 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:02.427033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:02.427703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:02.433105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:02.433560 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:02.434346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:02.434578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:02.434793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:02.434921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:02.435627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:02.436061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:02.636302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:02.641588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:02.645336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d9000bf-6283-43a2-a3e6-f475b80c50aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.225170 May 23 16:20:02.645918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1933/3973] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:02.649654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-24e8df95-8a6b-4954-9dec-8bdded032d81 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.226494 May 23 16:20:02.650219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2041/3974] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:03.663223 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:03.663923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:03.668396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:03.669153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:03.671465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:03.671916 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:03.672679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:03.673121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:03.678972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:03.679491 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:03.681530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:03.683509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:03.867564 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:03.870792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:03.875563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3b1660f8-fe0a-438d-bdd1-347de1365bac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216316 May 23 16:20:03.876129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1934/3975] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:03.879325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf4fd165-e9e6-451d-94ed-c9bed6bbcad5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215455 May 23 16:20:03.879885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2042/3976] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:04.893067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:04.893708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:04.896329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:04.896962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:04.900781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:04.901235 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:04.902253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:04.902694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:04.903498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:04.903959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:04.904713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:04.905151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:04.906583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:05.095119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:05.098615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:05.103800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8b5aabc1-0173-4f04-b07b-32e765c5e9c4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214638 May 23 16:20:05.104370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1935/3977] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:05.106683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e0c81b0-d9fb-464b-b177-ebac290f16f6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214656 May 23 16:20:05.107274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2043/3978] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:06.121133 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:06.121730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:06.122912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:06.123556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:06.128299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:06.128750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:06.129567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:06.129717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:06.130004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:06.130114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:06.130868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:06.131328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:06.323148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:06.326166 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:06.331634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-024af7ff-0270-4d6d-82f0-eec057378846 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212304 May 23 16:20:06.332911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2044/3979] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:06.334918 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f2bafdf8-5090-4b31-89fe-8899fa75bb92 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.217620 May 23 16:20:06.335552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1936/3980] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:07.349580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:07.350201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:07.352048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:07.352666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:07.356792 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:07.357250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:07.358025 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:07.358467 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:07.358996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:07.359484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:07.360254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:07.360695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:07.549956 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:07.554276 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:07.558638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ad267236-3178-4c94-a9a5-5e6a340baccd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213363 May 23 16:20:07.559247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2045/3981] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:07 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:07.562407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ced73d4-143c-4f08-a362-4b7ed24fef19 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214565 May 23 16:20:07.563134 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1937/3982] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:07 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:08.575577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:08.576129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:08.579914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:08.580526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:08.582837 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:08.583337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:08.584113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:08.584555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:08.586886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:08.587362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:08.588115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:08.588550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:08.774322 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:08.781288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:08.782635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1220d3c4-5ab5-4701-8889-a64384f4ed88 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210953 May 23 16:20:08.783239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2046/3983] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:08.789199 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ef8b01e8-ee63-4d99-9a34-a91ec467b1bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213301 May 23 16:20:08.789753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1938/3984] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:09.799391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:09.799990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:09.805727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:09.806351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:09.806509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:09.806963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:09.807755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:09.808192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:09.813119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:09.813579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:09.814351 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:09.815076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:10.006380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:10.010252 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:10.014412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8a523d27-4cc6-43ce-82a8-d11266851670 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218894 May 23 16:20:10.014989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2047/3985] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:10.018204 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e9ee983f-953a-461e-b12e-eec9a7599026 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216252 May 23 16:20:10.018756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1939/3986] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:11.032605 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:11.033219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:11.035717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:11.036350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:11.040395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:11.040847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:11.041610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:11.042045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:11.042776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:11.043258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:11.044023 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:11.044465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:11.233153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:11.236890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:11.243168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-de3c0faa-1a8d-428e-898a-a5cb905a3446 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214409 May 23 16:20:11.243747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2048/3987] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:11.244927 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b908b6b9-a53f-441d-aabe-4e37f10f7650 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213703 May 23 16:20:11.245489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1940/3988] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:12.260915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:12.261534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:12.262081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:12.262698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:12.268033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:12.268480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:12.269027 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:12.269273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:12.269477 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:12.269675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:12.270229 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:12.270664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:12.461987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:12.466259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:12.470204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a08658a7-ecc6-454e-b168-2aeda4a967e7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213094 May 23 16:20:12.470779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2049/3989] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:12 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:12.474161 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-457c5204-88a6-4cda-a162-5f91716618ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215873 May 23 16:20:12.474723 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1941/3990] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:13.487849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:13.489008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:13.491022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:13.491669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:13.495647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:13.496097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:13.496866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:13.497302 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:13.498062 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:13.498510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:13.499386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:13.499822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:13.694162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:13.698454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:13.706769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d7728e04-6d21-41ae-8c66-deafb2ee6cab tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.222870 May 23 16:20:13.707406 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2050/3991] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:13.710374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7a761765-583b-4f11-8329-b38038b7ef7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.223905 May 23 16:20:13.710940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1942/3992] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:14.725501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:14.726056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:14.727283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:14.727913 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:14.732507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:14.732959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:14.733709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:14.734139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:14.734249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:14.734675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:14.735470 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:14.735911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:14.930056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:14.933410 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:14.938054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fdb896fb-315a-443f-bba8-da5df947f321 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215157 May 23 16:20:14.938618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1943/3993] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:14.941170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6aabea45-6804-4ad3-809f-b671ecc3b67e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.220309 May 23 16:20:14.941728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2051/3994] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:15.956112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:15.956725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:15.959060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:15.959779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:15.963909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:15.964363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:15.965137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:15.965579 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:15.966386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:15.966841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:15.967677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:15.968120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:15.985845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:20:15.986236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:20:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 (ip 10.222.0.32) !!! May 23 16:20:16.183834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:16.188314 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:16.192099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-70ff64d4-5301-489f-bf97-93ca19542b7e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.240084 May 23 16:20:16.192675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1944/3995] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:16.196628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-236420c4-994c-4979-aec4-7e484e51e20f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.242231 May 23 16:20:16.197257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2052/3996] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:17.209795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:17.210417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:17.213721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:17.214340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:17.218049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:17.218591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:17.219436 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:17.219979 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:17.220685 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:17.221138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:17.221892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:17.222321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:17.410285 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:17.414840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:17.418451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eb1375ec-729c-4601-a96e-ec48460b1762 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213092 May 23 16:20:17.419404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1945/3997] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:17 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:17.422871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5ddf1ca0-2cd3-4903-979f-509fa6b6234a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213050 May 23 16:20:17.423530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2053/3998] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:17 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:18.437437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:18.438105 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:18.441489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:18.442142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:18.444641 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:18.445095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:18.445864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:18.446304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:18.448565 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:18.449018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:18.449779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:18.450219 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:18.454141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:18.641474 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:18.644304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:18.650028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dfc338e9-aa37-4b60-aeb7-cdded5aea1ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211776 May 23 16:20:18.650596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2054/3999] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:18.653031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c368eec-6f8c-4b55-b923-5ece8556e77c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219508 May 23 16:20:18.653597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1946/4000] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:19.667468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:19.668140 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:19.669447 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:19.669936 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:19.674886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:19.675694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:19.676293 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:19.676654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:19.676788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:19.677139 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:19.677474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:19.677904 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:19.870233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:19.877260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:19.878283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1f39f34c-4963-4dac-9b4c-4f69caf52594 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212693 May 23 16:20:19.879049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1947/4001] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:19.885341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-284288f4-dcb6-42b0-ab0c-8dfaee831dd8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.221927 May 23 16:20:19.890448 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2055/4002] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:20.896007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:20.896609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:20.904048 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:20.904503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:20.905267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:20.905708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:20.906028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:20.906640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:20.913148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:20.913589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:20.914375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:20.914844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:21.098670 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:21.101471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:21.106926 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-259437eb-2438-4ffa-ae34-e8c5869ef824 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214936 May 23 16:20:21.107611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1948/4003] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:21.109453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-77fd82ef-3a3b-472d-80e0-46cec8eb1ca8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207574 May 23 16:20:21.110126 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2056/4004] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:22.124487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:22.125082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:22.127416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:22.128039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:22.131557 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:22.132000 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:22.132757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:22.133188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:22.134429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:22.135494 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:22.136257 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:22.136693 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:22.321935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:22.329001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:22.330675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-59920319-70a7-45a2-bf97-4b949926df89 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210249 May 23 16:20:22.331294 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1949/4005] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:22 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:22.336889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-16de34f1-e966-4b45-a62c-0b0925e7a25b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213352 May 23 16:20:22.337445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2057/4006] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:22 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:23.347301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:23.347923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:23.353275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:23.353920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:23.354581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:23.355033 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:23.356343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:23.356786 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:23.360755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:23.361213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:23.361953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:23.362382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:23.548035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:23.554647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:23.556303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-33574d9c-5429-4eca-8235-21d87f634107 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212956 May 23 16:20:23.557086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1950/4007] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:23.563073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b9433050-747d-4a54-9800-4cd7f8df956c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213581 May 23 16:20:23.563655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2058/4008] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:24.574810 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:24.575410 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:24.578898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:24.579661 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:24.583742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:24.584192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:24.584971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:24.585413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:24.587408 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:24.587865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:24.588831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:24.589275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:24.777435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:24.780930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:24.784225 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:24.785746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0823f2b5-b17d-463e-b441-34d40b3135a2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215079 May 23 16:20:24.786324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1951/4009] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:24.792469 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-90355d5a-dd62-4801-aeb9-aa059809a594 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217574 May 23 16:20:24.793619 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2059/4010] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:25.803141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:25.803753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:25.809552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:25.810186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:25.810314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:25.810756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:25.811547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:25.811987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:25.816532 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:25.816978 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:25.817756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:25.818186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:26.243163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:26.248171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:26.252174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-66a50ec8-0817-4cf2-bd7e-a8d566d470bf tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.452923 May 23 16:20:26.252836 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1952/4011] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:26.256213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-027dcb6e-d8db-462e-bd0e-8664276d027c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.450526 May 23 16:20:26.256769 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2060/4012] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:27.270432 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:27.270981 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:27.273371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:27.273999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:27.278464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:27.278912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:27.279698 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:27.280129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:27.282069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:27.282069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:27.282844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:27.283308 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:27.474567 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:27.478196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:27.483218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bf45efb7-c1e9-41b2-8a61-37b3b4d815ac tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216641 May 23 16:20:27.483993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1953/4013] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:27 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:27.488775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-80b481fa-08de-412f-98fa-79bf84baa7dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219177 May 23 16:20:27.489433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2061/4014] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:27 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:28.500842 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:28.501380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:28.503715 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:28.505775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:28.506393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:28.508487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:28.508931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:28.509692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:28.510130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:28.512966 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:28.514891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:28.514891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:28.514891 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:28.700243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:28.706986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:28.708303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c71a24b3-a6c4-4efb-b970-1a02b50eb219 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211489 May 23 16:20:28.708878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1954/4015] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:28.714928 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-606a4d40-8523-4677-891d-4154c8eabbed tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212866 May 23 16:20:28.716019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2062/4016] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:29.725866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:29.726417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:29.732831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:29.733607 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:29.733752 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:29.734058 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:29.734822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:29.735286 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:29.737293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:29.742599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:29.743046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:29.743829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:29.744261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:29.931626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:29.934706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:29.939960 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-809f2c9a-3676-46ab-84d1-08240d931bd0 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218014 May 23 16:20:29.940536 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1955/4017] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:29.942598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1df4eac0-174d-46ce-8595-93037703c8cb tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214092 May 23 16:20:29.943178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2063/4018] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:30.956354 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:30.957007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:30.959174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:30.959807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:30.963593 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:30.964050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:30.964822 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:30.965269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:30.966108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:30.966560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:30.967355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:30.967983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:31.171663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:31.179693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-32f7bb0e-732e-46ba-b818-4e018db0b541 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.227251 May 23 16:20:31.180274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1956/4019] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:31.181688 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:31.190180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7efeb384-3f75-4ee2-82e3-fb09dd05808d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.234909 May 23 16:20:31.190736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2064/4020] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:32.197258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:32.197787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:32.204278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:32.204730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:32.205489 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:32.205924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:32.207380 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:32.207990 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:32.214711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:32.215698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:32.216452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:32.216882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:32.401039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:32.405641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:32.409742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dcaaaf37-1400-40e7-a114-f6484a9e4dfa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216433 May 23 16:20:32.410317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1957/4021] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:32 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:32.413677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-680a7717-7b06-4412-b014-30947f323119 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210073 May 23 16:20:32.414234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2065/4022] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:32 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:33.430222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:33.430842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:33.438739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:33.439407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:33.440362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:33.440896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:33.501044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:33.501669 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:33.507823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:33.508277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:33.509197 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:33.509633 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:33.620022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:33.628643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8329b6b8-6bcc-4819-9ebb-f39eaa27e511 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.202631 May 23 16:20:33.629218 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2066/4023] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:33.664005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:33.671869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-34b427e4-1d1d-4cb8-b14f-629a6bf4009c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.248991 May 23 16:20:33.672464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1958/4024] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:34.646504 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:34.647023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:34.654339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:34.654786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:34.655574 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:34.656012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:34.688755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:34.689377 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:34.695908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:34.696365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:34.697149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:34.697594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:34.830236 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:34.838229 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-56370a3d-e448-4472-8b22-c59a5706e14b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.195776 May 23 16:20:34.838805 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2067/4025] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:34 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:34.862009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:34.869816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d2fe923f-7257-4686-bd6e-bdc2016ce6ef tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.184859 May 23 16:20:34.870376 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1959/4026] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:34 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:35.856694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:35.857315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:35.863334 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:35.863784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:35.864541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:35.864980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:35.886759 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:35.887419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:35.893834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:35.894279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:35.895245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:35.895717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:36.040743 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:36.049266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bcd0d90d-9ef2-4808-b70d-e82b2419cdd6 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.197215 May 23 16:20:36.050051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2068/4027] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:35 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:36.061375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:36.069240 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-13dbb32b-8343-4b30-bf60-6b19d8e8cbe2 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.186279 May 23 16:20:36.069800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1960/4028] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:35 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:37.066264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:37.066880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:37.073318 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:37.073768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:37.074523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:37.074959 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:37.086637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:37.087299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:37.093690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:37.094137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:37.094894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:37.095358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:37.282223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:37.290337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d4eb6e9c-4baa-4f80-b07a-6568017574dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.227944 May 23 16:20:37.293705 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2069/4029] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:37 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:37.293910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:37.302243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7729f133-0706-48fd-957c-a8770a138a1f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219460 May 23 16:20:37.302835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1961/4030] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:37 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:38.308027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:38.309414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:38.315970 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:38.316422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:38.317179 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:38.317614 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:38.319137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:38.319944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:38.326103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:38.326547 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:38.327325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:38.327757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:38.513596 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:38.516680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:38.517008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:38.521803 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-642dd06a-1a52-4727-a4fa-821834a39c6d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217684 May 23 16:20:38.522376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2070/4031] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:38 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:38.524626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5f3a8a26-c268-45c5-8289-af6d31ef5c3e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209258 May 23 16:20:38.525178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1962/4032] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:38 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:39.539601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:39.540174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:39.541402 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:39.542015 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:39.546742 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:39.547220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:39.547989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:39.548358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:39.548490 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:39.548806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:39.549555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:39.549987 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:39.742186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:39.745610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:39.750919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1cc5de4a-2ce2-4c40-b37e-c519b89016f4 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213238 May 23 16:20:39.751583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1963/4033] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:39 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:39.754153 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0a5eb51d-3716-4819-b138-45bc39260e12 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218959 May 23 16:20:39.754779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2071/4034] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:39 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:40.768556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:40.769159 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:40.771864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:40.772531 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:40.775728 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:40.776181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:40.776931 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:40.778049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:40.779566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:40.780010 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:40.780764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:40.781192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:40.972481 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:40.976414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:40.980606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-be6f0664-3ee1-45f6-b0a0-78943f1aeb13 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215862 May 23 16:20:40.981165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1964/4035] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:40 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:40.984378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ced95ea8-a582-4a9f-9ecc-7c304013cf41 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216272 May 23 16:20:40.984931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2072/4036] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:40 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:41.997689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:41.998342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:42.000862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:42.001479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:42.004982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:42.005442 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:42.006211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:42.006658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:42.007764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:42.008239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:42.008989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:42.009422 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:42.198876 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:42.202303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:42.207469 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8421d0cf-5f1c-4fbe-9b5e-455fd7d4350d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213555 May 23 16:20:42.208268 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1965/4037] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:41 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:42.210258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3d1a77d2-e894-4d31-8ea5-ec1a1370d8dd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213238 May 23 16:20:42.210809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2073/4038] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:41 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:43.224479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:43.225104 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:43.227194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:43.227884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:43.232130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:43.232589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:43.233368 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:43.233809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:43.234626 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:43.235072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:43.235853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:43.236283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:43.416908 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:43.424919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-58a3826e-e8cc-4ae2-a0d2-8d05892c122c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.204249 May 23 16:20:43.425748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1966/4039] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:43 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:43.427836 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:43.436402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5f4f54db-b95f-4bd6-bdc9-ffa6f2f03177 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212748 May 23 16:20:43.436960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2074/4040] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:43 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:44.443175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:44.443594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:44.449864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:44.450352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:44.451150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:44.451590 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:44.452457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:44.453105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:44.459522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:44.459972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:44.460725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:44.461159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:44.644694 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:44.647768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:44.653272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7cc4917-6aa9-4a92-ade0-012722bddeb7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214289 May 23 16:20:44.653843 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1967/4041] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:44 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:44.656451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6fd8e682-e9fb-4a61-a4d9-c85d9d99bec9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207284 May 23 16:20:44.657004 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2075/4042] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:44 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:45.672265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:45.672886 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:45.673589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:45.674235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:45.679449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:45.679901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:45.680569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:45.680750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:45.681014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:45.681155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:45.681765 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:45.682192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:45.699384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:45.883727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:45.891981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:45.892531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2420c2aa-00f6-4451-9b9f-95cc7250762d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.224701 May 23 16:20:45.893126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1968/4043] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:45 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:45.902162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-72f57cfe-9cdc-4ab3-bd98-4d042cf89dc3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.232884 May 23 16:20:45.903014 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2076/4044] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:45 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:46.910238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:46.910921 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:46.917398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:46.917845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:46.918599 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:46.919032 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:46.919756 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:46.920373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:46.927472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:46.927923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:46.928681 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:46.929118 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:47.108693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:47.113138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:47.117424 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ec7b43a7-54ed-449a-95b3-74d1a8622454 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211485 May 23 16:20:47.118001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1969/4045] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:46 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:47.121030 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cfaad6de-928b-4019-b04d-98f2f61f94d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.205058 May 23 16:20:47.121584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2077/4046] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:46 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:48.134393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:48.135005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:48.137407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:48.138466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:48.141679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:48.142121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:48.142877 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:48.143336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:48.144434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:48.144883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:48.145636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:48.146071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:48.331487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:48.337999 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:48.339583 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f98aefa9-02f8-4f3e-8338-ecf738a0b79c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208993 May 23 16:20:48.340238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1970/4047] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:48 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:48.346192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-79ba3df2-efcb-410c-94c1-a4320a2e4405 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212559 May 23 16:20:48.346746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2078/4048] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:48 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:49.356826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:49.357420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:49.362589 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:49.363246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:49.363911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:49.364394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:49.365163 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:49.365600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:49.369576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:49.370018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:49.370768 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:49.371228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:49.556349 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:49.559456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:49.565338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-04629cf6-114b-426e-b606-4f1d5d5f3d36 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212438 May 23 16:20:49.565914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1971/4049] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:49 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:49.568215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1628b4c1-883d-417b-90d3-acc29ce90174 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.209520 May 23 16:20:49.568771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2079/4050] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:49 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:50.584067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:50.584621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:50.585095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:50.585710 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:50.591173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:50.591695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:50.592462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:50.592899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:50.593330 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:50.594402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:50.595207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:50.595649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:50.805784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:50.809432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:50.815821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-941f407f-5472-4f3a-9c32-5df7b0db93ba tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.234650 May 23 16:20:50.820089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1972/4051] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:50 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:50.820926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3c7e76d7-f412-4774-84b0-426585404b00 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.239840 May 23 16:20:50.821507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2080/4052] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:50 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:51.832233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:51.832791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:51.839428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:51.839907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:51.840682 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:51.840871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:51.841312 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:51.841484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:51.847856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:51.848300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:51.849079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:51.849523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:52.036788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:52.039784 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:52.044717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ac3753c5-282c-4972-be7a-5fc64808ccd9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216483 May 23 16:20:52.045393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1973/4053] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:51 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:52.048125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-587b2f3e-3006-492d-bede-18e88d832f66 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211061 May 23 16:20:52.048682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2081/4054] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:51 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:53.063144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:53.063775 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:53.065093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:53.065706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:53.070399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:53.070856 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:53.071689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:53.072158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:53.072292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:53.072599 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:53.073353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:53.073783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:53.260950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:53.264901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:53.268964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-423216f3-b1c7-4d24-b61a-c07016456dc9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210114 May 23 16:20:53.269540 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1974/4055] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:53 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:53.273073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f1bbba8d-a9cf-4cd8-a784-88d2733e5379 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212234 May 23 16:20:53.274346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2082/4056] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:53 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:54.286784 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:54.287419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:54.289428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:54.290046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:54.293903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:54.294364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:54.295114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:54.295574 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:54.296397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:54.296847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:54.297604 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:54.298233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:54.485098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:54.488917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:54.493096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0c59dbcd-16ec-44d5-a88a-9aa339c4c039 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210112 May 23 16:20:54.493659 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1975/4057] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:54 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:54.496872 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e7fb24fa-9eed-494a-bf41-6d03738398ea tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211308 May 23 16:20:54.497423 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2083/4058] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:54 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:55.510288 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:55.510919 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:55.512824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:55.513447 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:55.517089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:55.517544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:55.518311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:55.518748 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:55.519840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:55.520288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:55.521046 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:55.521485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:55.706685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:55.713561 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:55.714763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-22dadcac-15ed-45e7-a1d2-b51fa3bafe8e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208533 May 23 16:20:55.715454 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1976/4059] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:55 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:55.721512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b8a98e36-40ae-4241-8dfc-50d12a846c75 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212478 May 23 16:20:55.722068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2084/4060] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:55 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:56.731780 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:56.732407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:56.738015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:56.738657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:56.739070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:56.739555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:56.740320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:56.740758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:56.745245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:56.745774 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:56.746547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:56.746992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:56.932295 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:56.936202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:56.941076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1406516b-d4f0-4743-822d-f6a0e2faad71 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213206 May 23 16:20:56.942506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1977/4061] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:56 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:56.945325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-438a124c-a475-45de-b365-d976743e44b5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211140 May 23 16:20:56.945883 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2085/4062] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:56 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:57.958039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:57.958767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:57.962316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:57.963041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:57.965298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:57.965743 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:57.966732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:57.967200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:57.969446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:57.969892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:57.970642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:57.971073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:58.148812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:58.157526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9a33cc60-1947-41df-ae0b-0fc6490cbb41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.203365 May 23 16:20:58.158090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1978/4063] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:57 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:58.163389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:58.171389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7852857e-83f2-44c5-b226-7718aada1b7a tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212840 May 23 16:20:58.171997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2086/4064] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:57 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:20:59.174538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:59.175169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:59.177423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=50 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:59.182121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:59.183666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:59.183666 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:59.183898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:59.189136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:20:59.192770 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:20:59.196357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:59.196812 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:59.197575 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:20:59.198015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:20:59.216347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:20:59.610437 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:59.618322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f580adf9-6151-4f25-8749-996a74ecaf41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.447596 May 23 16:20:59.618891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1979/4065] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:59 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:20:59.631683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:20:59.639649 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-00ee66ee-5a22-4203-8a0a-ad9674966047 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.454890 May 23 16:20:59.640213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2087/4066] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:20:59 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:00.636149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:00.636719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:00.643175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:00.643626 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:00.644379 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:00.644815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:00.657863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:00.658472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:00.660757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:00.665434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:00.675751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:00.675751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:00.675751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:01.020380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:01.025051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:01.029852 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-73e259fb-a6ab-41ca-b6df-c89a88988e12 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.397607 May 23 16:21:01.030435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1980/4067] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:00 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:01.033910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c96ebbef-8ef9-41ed-b6a7-bb5db0bb590b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.380612 May 23 16:21:01.034556 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2088/4068] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:00 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:02.047501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:02.048255 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:02.050352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:02.050997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:02.054851 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:02.055347 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:02.056113 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:02.056550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:02.057310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:02.057753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:02.058498 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:02.058926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:02.248584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:02.252185 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:02.257297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ae15a85-94cd-4983-b00e-19fdce131964 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210651 May 23 16:21:02.257952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2089/4069] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:02 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:02.260091 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9d8b10ff-f0b5-4e7b-b91a-195078c1b118 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216521 May 23 16:21:02.260647 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1981/4070] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:02 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:03.273728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:03.274354 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:03.277135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:03.277751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:03.281816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:03.282279 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:03.283039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:03.283501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:03.285130 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:03.285577 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:03.286329 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:03.286760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:03.480712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:03.483698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:03.489961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-51d09be1-f02f-44c9-84c7-89d1bcf9a9c6 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215395 May 23 16:21:03.489961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1982/4071] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:03 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:03.491502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-834ff7ca-d65a-42c8-9b75-5afb1cee9d21 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.221676 May 23 16:21:03.492054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2090/4072] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:03 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:04.506522 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:04.507195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:04.508509 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:04.509121 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:04.513865 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:04.514314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:04.515118 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:04.516018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:04.516260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:04.516466 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:04.517223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:04.517659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:04.713882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:04.716902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:04.723328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d482aa43-fd7e-4a4b-aedd-042296ec475e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218530 May 23 16:21:04.723919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2091/4073] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:04 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:04.725450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b99976ff-fdda-47a2-949e-d2e32470f99e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.222836 May 23 16:21:04.726003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1983/4074] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:04 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:05.740165 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:05.740757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:05.741396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:05.742022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:05.747289 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:05.747737 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:05.748487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:05.748652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:05.748925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:05.749109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:05.749881 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:05.750350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:05.952672 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:05.957344 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:05.961284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-682da2e0-5001-454a-9392-eab46e778b58 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.224933 May 23 16:21:05.961885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2092/4075] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:05 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:05.965326 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-93431d46-ddc8-48d8-aee0-c51434966fde tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.227656 May 23 16:21:05.965891 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1984/4076] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:05 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:06.977913 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:06.978464 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:06.981548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:06.982168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:06.985061 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:06.985519 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:06.986283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:06.986719 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:06.988690 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:06.989144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:06.989901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:06.990341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:06.994243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:07.179762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:07.182991 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:07.187758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1c375525-beeb-446c-bd07-4a76391118d9 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213824 May 23 16:21:07.188415 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2093/4077] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:06 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:07.191671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e37220bb-ae91-447a-b6e9-9b10ff742b6f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213980 May 23 16:21:07.192336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1985/4078] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:06 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:08.206170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:08.206813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:08.208894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:08.209503 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:08.213337 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:08.213788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:08.214549 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:08.214988 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:08.215827 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:08.216273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:08.217020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:08.217451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:08.412893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:08.415655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:08.421119 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f386e705-cb48-4436-9e64-4415b51e8e3f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.219445 May 23 16:21:08.423144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2094/4079] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:08 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:08.423814 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-be1175cc-8523-44d4-a65a-978b6dc3a6e5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218610 May 23 16:21:08.424382 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1986/4080] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:08 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:09.438240 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:09.438985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:09.440494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:09.441164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:09.445629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:09.446076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:09.446835 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:09.447315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:09.447673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:09.448123 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:09.448874 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:09.449309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:09.637249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:09.641129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:09.645639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c9705136-4a1b-401e-bf4b-5ac8a97779c4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211323 May 23 16:21:09.646304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2095/4081] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:09 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:09.649126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a4821364-e26a-44c3-a342-59e038f36382 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212391 May 23 16:21:09.649679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1987/4082] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:09 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:10.663137 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:10.663702 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:10.665463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:10.666089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:10.670157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:10.670602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:10.671409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:10.671843 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:10.672519 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:10.673483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:10.674244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:10.674688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:10.867216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:10.870433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:10.875341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8e465937-9f06-41e0-b211-5e7d7233c3a7 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216154 May 23 16:21:10.876006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2096/4083] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:10 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:10.878433 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-027e97bb-c2f9-4436-b1cf-f736280b5193 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216747 May 23 16:21:10.878995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1988/4084] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:10 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:11.892783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:11.893357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:11.894955 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:11.895661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:11.900951 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:11.901690 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:11.902037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:11.902495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:11.902981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:11.903290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:11.903818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:11.903990 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:12.108350 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:12.111745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:12.117001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1a916033-3568-4d4f-941a-28bdc38e7300 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.226394 May 23 16:21:12.117813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1989/4085] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:11 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:12.121634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2ded0507-d15e-418c-b2fc-1d09ceaa41dc tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.233293 May 23 16:21:12.122328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2097/4086] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:11 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:13.134707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:13.135365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:13.139471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:13.140076 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:13.142479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:13.142938 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:13.143742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:13.144186 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:13.146569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:13.147011 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:13.147794 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:13.148230 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:13.333755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:13.337864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:13.341800 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e6dc362a-f971-4006-b5a5-8f8c02f5a950 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.210954 May 23 16:21:13.342372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1990/4087] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:13 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:13.345907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-254112a1-912e-49b5-bc20-40c3b7ccb295 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210714 May 23 16:21:13.346456 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2098/4088] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:13 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:14.358718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:14.359378 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:14.361818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:14.362437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:14.366022 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:14.366473 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:14.367272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:14.367719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:14.368948 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:14.369405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:14.370169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:14.370612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:14.555045 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:14.562033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:14.562944 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bb6471be-ff81-4074-95ee-15cfb67039ff tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.208057 May 23 16:21:14.564094 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1991/4089] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:14 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:14.570453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-94eb8fe0-93ec-4d0a-8153-31f527892504 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212419 May 23 16:21:14.571006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2099/4090] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:14 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:15.581028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:15.581597 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:15.587691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:15.588125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:15.588312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:15.588455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:15.589211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:15.589645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:15.594642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:15.595634 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:15.596398 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:15.596838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:15.785530 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:15.788481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:15.793563 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-467395f7-3b3f-4c17-b875-2fc35e19ac48 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.216526 May 23 16:21:15.794151 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1992/4091] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:15 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:15.796553 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6e99d058-e4b3-449e-a0d6-8c44900fb428 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.212763 May 23 16:21:15.797106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2100/4092] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:15 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:16.809755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:16.810517 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:16.813122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:16.813788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:16.817581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:16.818031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:16.818795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:16.819257 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:16.820788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:16.821242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:16.822006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:16.822454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:17.010090 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:17.013317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:17.018296 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-df39ceff-6efa-4158-b913-c32176b06549 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212409 May 23 16:21:17.018965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1993/4093] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:16 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:17.021213 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b70724e7-8450-4afd-9ccb-dc00b73dd34e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.211836 May 23 16:21:17.021766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2101/4094] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:16 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:18.035084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:18.038001 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:18.038159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:18.038560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:18.042316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:18.042777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:18.043571 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:18.044036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:18.045008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:18.045462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:18.046227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:18.046664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:18.236674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:18.241112 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:18.242543 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fd97879c-1954-4cab-b7a9-2dcebd555430 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211267 May 23 16:21:18.243148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1994/4095] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:18 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:18.249060 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9e49a740-415e-4c34-8cff-3d2a4da9b2ac tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.214915 May 23 16:21:18.249610 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2102/4096] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:18 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:19.259755 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:19.260587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:19.265388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:19.266002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:19.267132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:19.267591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:19.268360 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:19.268799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:19.272434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:19.272888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:19.273673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:19.274108 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:19.706279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:19.710283 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:19.714592 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f78c90e4-424c-4e20-9ce1-9c0510918614 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.458696 May 23 16:21:19.715273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1995/4097] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:19 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:19.718209 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cc03d50a-14c3-4e5e-8235-63cbb5a253d5 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.456509 May 23 16:21:19.719087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2103/4098] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:19 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:20.732220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:20.732840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:20.734880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:20.736101 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:20.740047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:20.740506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:20.741274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:20.741711 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:20.742480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:20.742937 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:20.743738 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:20.744183 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:20.748105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:20.929419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:20.936288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:20.937409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-01eccffb-e4cc-4d9a-8859-f8d897f9a70a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209014 May 23 16:21:20.938020 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1996/4099] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:20 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:20.944523 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31859775-7474-42e7-82e8-0fa018247e08 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213422 May 23 16:21:20.945075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2104/4100] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:20 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:21.954585 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:21.955174 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:21.961783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:21.962260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:21.962433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:21.962735 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:21.963529 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:21.963973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:21.968844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:21.969299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:21.970059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:21.970497 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:22.162184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:22.165628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:22.170399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a408e601-6ac6-455b-8236-663062ab98aa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219754 May 23 16:21:22.171324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1997/4101] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:21 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:22.173697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6df76f35-9f51-4180-9157-a06e59818001 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215791 May 23 16:21:22.174252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2105/4102] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:21 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:23.187961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:23.188584 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:23.189576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:23.190201 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:23.195817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:23.196274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:23.197047 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:23.197181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:23.197495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:23.197628 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:23.198287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:23.198728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:23.393795 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:23.397450 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:23.403907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1ff4bc7a-ca27-489d-a5e2-06b99e4f7a00 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.219632 May 23 16:21:23.404491 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1998/4103] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:23 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:23.406044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c0e23b7f-9292-4ead-8e58-b3aa7cbda211 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.220150 May 23 16:21:23.407224 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2106/4104] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:23 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:24.421468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:24.422119 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:24.423352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:24.424107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:24.428389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:24.428838 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:24.429587 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:24.430018 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:24.430457 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:24.430909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:24.431704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:24.432147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:24.621726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:24.624655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:24.630531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6b4838df-6cb7-465e-be69-621fd8ebf4bc tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.212766 May 23 16:21:24.631025 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 1999/4105] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:24 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:24.633211 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-31a4747b-a44e-45a1-9abb-f503e4f413d3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.213618 May 23 16:21:24.633771 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2107/4106] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:24 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:25.647201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:25.647798 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:25.649547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:25.650170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:25.654320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:25.654883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:25.655761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:25.656217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:25.656830 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:25.657281 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:25.658035 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:25.658471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:25.850661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:25.853911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:25.858657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36971d81-e8e1-48d5-9ac1-f4db3b9ee42e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215439 May 23 16:21:25.859332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2000/4107] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:25 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:25.861886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2e6d3e65-7232-4c90-9ced-b0417e953cfd tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216272 May 23 16:21:25.862428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2108/4108] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:25 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:26.876390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:26.876941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:26.879478 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:26.880093 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:26.884052 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:26.884534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:26.885301 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:26.885740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:26.886564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:26.887070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:26.887860 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:26.888300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:27.082364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:27.085169 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:27.085859 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=100 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:27.090681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ba662e0f-6973-431b-b7cc-46f228e5e1b8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.218268 May 23 16:21:27.091277 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2001/4109] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:26 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:27.093192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b00802f5-1064-4ac7-9833-c62e1e61f823 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.218076 May 23 16:21:27.093761 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2109/4110] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:26 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:28.107791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:28.108414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:28.113540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:28.114157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:28.114998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:28.115483 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:28.116254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:28.116695 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:28.120776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:28.121232 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:28.121997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:28.122433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:28.308777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:28.312087 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:28.317474 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7c10435-18ef-40b4-bb56-774487af90a9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.214035 May 23 16:21:28.318135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2002/4111] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:28 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:28.320473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-70f59e64-e4cd-4dd0-9320-1080b8fc1d8d tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.210750 May 23 16:21:28.321190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2110/4112] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:28 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:29.334769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:29.335449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:29.337487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:29.337948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=48 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:29.338862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:29.342628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:29.343224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:29.343815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:29.344311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:29.345759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:29.346221 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:29.346997 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:29.347476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:29.531418 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:29.539399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e75f08fe-4fa3-4670-b63f-a3c63befff7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.209093 May 23 16:21:29.540070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2003/4113] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:29 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:29.542205 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:29.550431 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-62adf432-8759-4b16-96fd-5c66df15ea03 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.216723 May 23 16:21:29.550993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2111/4114] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:29 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:30.557016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:30.557575 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:30.564077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:30.564542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:30.565309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:30.565753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:30.566874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:30.567522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:30.573791 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:30.574234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:30.574986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:30.575451 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:30.758450 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:30.762639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:30.766531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8a669952-711d-41b1-89f8-eae2f290848d tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.213420 May 23 16:21:30.767121 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2004/4115] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:30 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:30.771122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3a0d5a8e-222d-4e3f-917c-1f5d1b5c7a6e tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.207996 May 23 16:21:30.771683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2112/4116] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:30 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:31.783943 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:31.784566 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:31.788362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:31.789069 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:31.791459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=50 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:31.791834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:31.792287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:31.793050 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:31.793497 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:31.796106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:31.801986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:31.801986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:31.801986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:31.987103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:31.991044 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:31.997371 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-97c9bdd7-174f-44e0-b61f-efda5641fc7f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.215659 May 23 16:21:31.998031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2005/4117] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:31 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:31.999086 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-16486994-1b48-4f2b-982e-9297c7aeb4e4 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.215047 May 23 16:21:31.999675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2113/4118] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:31 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:33.015726 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:33.016384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:33.016670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:33.017309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3e090a96-ef41-4548-ac75-e4dd2f371539] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:33.023083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:33.023569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:33.023715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:33.024079 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:33.024338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:33.024779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:33.024905 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:33.025244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:33.215440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:33.222110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:33.223462 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ad84eacb-b7bc-40c5-bf63-3524893301f8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 200 len: 1721 microversion: 2.1 time: 0.211554 May 23 16:21:33.224044 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2006/4119] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:33 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:33.229954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-08dd5b99-d914-4975-a74d-558f3c2181d1 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539" status: 200 len: 1737 microversion: 2.1 time: 0.217051 May 23 16:21:33.230660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2114/4120] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:33 2022] GET /compute/v2.1/servers/3e090a96-ef41-4548-ac75-e4dd2f371539 => generated 1737 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:21:33.240887 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6e980954-c127-4725-912a-b15d2671dbc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '.version_select at 0x7f47fea59d30>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:33.244212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR oslo.messaging._drivers.impl_rabbit [None req-6e980954-c127-4725-912a-b15d2671dbc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [ffd3eaeb-a204-45ff-ae87-297cfa49902c] AMQP server on 10.222.0.32:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 16:21:33.246146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-12f9ac47-7b79-41c5-815d-44e8094a8b73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '.version_select at 0x7f47f914b1f0>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:33.249412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR oslo.messaging._drivers.impl_rabbit [None req-12f9ac47-7b79-41c5-815d-44e8094a8b73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [f161fde2-2e97-49c3-ab70-5698f4f6f2a3] AMQP server on 10.222.0.32:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 23 16:21:34.251526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [None req-6e980954-c127-4725-912a-b15d2671dbc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [ffd3eaeb-a204-45ff-ae87-297cfa49902c] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 56868. May 23 16:21:34.256157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [None req-12f9ac47-7b79-41c5-815d-44e8094a8b73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [f161fde2-2e97-49c3-ab70-5698f4f6f2a3] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 56870. May 23 16:21:34.323506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6e980954-c127-4725-912a-b15d2671dbc5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1113968782" status: 202 len: 0 microversion: 2.1 time: 1.086694 May 23 16:21:34.324126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2007/4121] 10.222.0.32 () {58 vars in 1220 bytes} [Mon May 23 16:21:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1113968782 => generated 0 bytes in 1088 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:21:34.335845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-12f9ac47-7b79-41c5-815d-44e8094a8b73 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1932700632" status: 202 len: 0 microversion: 2.1 time: 1.093372 May 23 16:21:34.336472 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2115/4122] 10.222.0.32 () {58 vars in 1220 bytes} [Mon May 23 16:21:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1932700632 => generated 0 bytes in 1095 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 16:21:35.389069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-2f1b3e2c-cfb8-4494-b300-fcc83c5155e1 req-2bbb0b84-4435-4999-9ca7-96d0bfae2d23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:35.399393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2f1b3e2c-cfb8-4494-b300-fcc83c5155e1 req-2bbb0b84-4435-4999-9ca7-96d0bfae2d23 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:35.399946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-2f1b3e2c-cfb8-4494-b300-fcc83c5155e1 req-2bbb0b84-4435-4999-9ca7-96d0bfae2d23 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:35.412341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-2f1b3e2c-cfb8-4494-b300-fcc83c5155e1 req-2bbb0b84-4435-4999-9ca7-96d0bfae2d23 service nova] Creating event network-changed:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 16:21:35.418974 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-2f1b3e2c-cfb8-4494-b300-fcc83c5155e1 req-2bbb0b84-4435-4999-9ca7-96d0bfae2d23 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091535 May 23 16:21:35.419949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2008/4123] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:21:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:35.645492 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-f9e96d3b-e585-4ffd-8aa7-515981c67a73 req-5b423184-c7b2-4d0a-9b66-9bb9944843fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d808c02-6a3e-4e38-bfd1-986e31e8cbd2", "tag": "16595806-91fa-49b4-9688-d528f91f3cca"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:35.670535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f9e96d3b-e585-4ffd-8aa7-515981c67a73 req-5b423184-c7b2-4d0a-9b66-9bb9944843fd service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:35.671226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f9e96d3b-e585-4ffd-8aa7-515981c67a73 req-5b423184-c7b2-4d0a-9b66-9bb9944843fd service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:35.684142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-f9e96d3b-e585-4ffd-8aa7-515981c67a73 req-5b423184-c7b2-4d0a-9b66-9bb9944843fd service nova] Creating event network-changed:16595806-91fa-49b4-9688-d528f91f3cca for instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 on hv-cinder-83588 May 23 16:21:35.692286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-f9e96d3b-e585-4ffd-8aa7-515981c67a73 req-5b423184-c7b2-4d0a-9b66-9bb9944843fd service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.151705 May 23 16:21:35.693993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2116/4124] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:21:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:35.707164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6e226f86-80bc-422a-b7b9-7b0719784d5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: .version_select at 0x7f47fea59040>, body: {"keypair": {"name": "tempest-keypair-768906288", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJXA+70/CghRXojyAnSd1Wuw9WfgFADE+eM6qYzf6b/8PuoURykKzl2uw94jHTy+BoHwLUX9o2L8l2tEx2T4zJf/EZv18WsZyL0xbFSKSzd5xG8WxpqCRsG2gAGr8oIU3w=="}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:35.727332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-6e226f86-80bc-422a-b7b9-7b0719784d5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for project e80aa9835e9845e798b1c8346de141e3. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:21:35.731181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-6e226f86-80bc-422a-b7b9-7b0719784d5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for user 8b0a82159c6343d28546ec7c775b8791 and project e80aa9835e9845e798b1c8346de141e3. Resources: {'key_pairs'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:21:35.821789 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6e226f86-80bc-422a-b7b9-7b0719784d5c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.118704 May 23 16:21:35.822395 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2009/4125] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 16:21:35 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:37.982627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1453330291", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ec55f89b-6a09-430a-bad3-b52794f9cc56"}], "security_groups": [{"name": "tempest-securitygroup--198526746"}], "key_name": "tempest-keypair-768906288", "user_data": "}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:38.027625 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:21:38.028082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:21:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:21:38.090178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.network.neutron [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] validate_networks() for [('ec55f89b-6a09-430a-bad3-b52794f9cc56', None, None, None, None, None)] {{(pid=124966) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 16:21:38.494145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Flavor limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 16:21:38.494750 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Image limits 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 16:21:38.495258 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Flavor pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 16:21:38.495698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Image pref 0:0:0 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 16:21:38.496138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.virt.hardware [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124966) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 16:21:38.504875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:38.505851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:38.506595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:38.507027 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:38.523947 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for project e80aa9835e9845e798b1c8346de141e3. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:21:38.527864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Getting quotas for user 8b0a82159c6343d28546ec7c775b8791 and project e80aa9835e9845e798b1c8346de141e3. Resources: {'ram', 'instances', 'cores'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:21:38.538800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Going to run 1 instances... {{(pid=124966) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 16:21:38.541161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124966) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 16:21:38.633307 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5a61e427-f229-46ae-bc71-239e2b382023 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.655754 May 23 16:21:38.633954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2117/4126] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 16:21:37 2022] POST /compute/v2.1/servers => generated 401 bytes in 657 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 16:21:38.644399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-575a5c78-9649-4bba-90a3-8aa19028548f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:38.645551 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-575a5c78-9649-4bba-90a3-8aa19028548f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:38.703869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-575a5c78-9649-4bba-90a3-8aa19028548f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:38.711221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-575a5c78-9649-4bba-90a3-8aa19028548f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1237 microversion: 2.1 time: 0.071176 May 23 16:21:38.711769 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2010/4127] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:38 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1237 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:39.729032 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5419d7ec-084f-4e0b-8193-2801e69e0654 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:39.729583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5419d7ec-084f-4e0b-8193-2801e69e0654 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:39.784362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5419d7ec-084f-4e0b-8193-2801e69e0654 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:39.791677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5419d7ec-084f-4e0b-8193-2801e69e0654 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1237 microversion: 2.1 time: 0.067705 May 23 16:21:39.792250 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2118/4128] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:39 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1237 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:40.808782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-006cb3fb-1b74-4cfd-9204-3a969e9b431c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:40.809404 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-006cb3fb-1b74-4cfd-9204-3a969e9b431c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:40.860627 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-006cb3fb-1b74-4cfd-9204-3a969e9b431c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:40.868085 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-006cb3fb-1b74-4cfd-9204-3a969e9b431c tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1237 microversion: 2.1 time: 0.063159 May 23 16:21:40.868651 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2011/4129] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:40 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1237 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:41.884407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:41.885059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:41.891819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:41.892275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:41.893041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:41.893482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:41.956485 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:41.964454 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a704f2ea-bb2f-4796-860c-74ae3b14e6fa tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1241 microversion: 2.1 time: 0.084020 May 23 16:21:41.965019 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2119/4130] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:41 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1241 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:42.981431 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:42.995722 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:43.003510 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:43.003965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:43.004739 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:43.005191 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:43.073864 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:43.081279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c9fa5e82-5702-4cdd-9ee7-6255792e2825 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1307 microversion: 2.1 time: 0.103667 May 23 16:21:43.081875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2012/4131] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:42 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1307 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:44.100541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:44.101152 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:44.107583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:44.115811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:44.115811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:44.115811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:44.341167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:44.349074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0ef689c3-2dbc-45af-9a3e-4823ec1bd7c7 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.253216 May 23 16:21:44.349766 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2120/4132] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:44 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:44.549581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-18358004-5ab8-48f1-b397-5e8bae817a11 req-4eebfe08-7730-486c-9c40-988e26fa864d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ad71e05-689f-42a2-8cf7-82c2a3562e9d", "tag": "8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:44.561136 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-18358004-5ab8-48f1-b397-5e8bae817a11 req-4eebfe08-7730-486c-9c40-988e26fa864d service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:44.561589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-18358004-5ab8-48f1-b397-5e8bae817a11 req-4eebfe08-7730-486c-9c40-988e26fa864d service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:44.573149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-18358004-5ab8-48f1-b397-5e8bae817a11 req-4eebfe08-7730-486c-9c40-988e26fa864d service nova] Creating event network-changed:8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0 for instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d on hv-cinder-83588 May 23 16:21:44.581923 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-18358004-5ab8-48f1-b397-5e8bae817a11 req-4eebfe08-7730-486c-9c40-988e26fa864d service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035955 May 23 16:21:44.582537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2013/4133] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:21:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:45.365462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:45.366089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:45.372715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:45.373177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:45.377248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:45.377248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:45.501263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:45.509138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-89ddabc1-76e8-42ac-8448-87a1ff1db612 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.147557 May 23 16:21:45.509704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2121/4134] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:45 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:46.527732 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:46.528461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:46.534920 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:46.535403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:46.536164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:46.536601 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:46.662586 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:46.670075 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6b9e75d7-c31d-41bb-9eb0-e6e522891a80 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.147220 May 23 16:21:46.670638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2014/4135] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:46 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:47.687321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:47.687972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:47.694779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:47.695263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:47.696022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:47.696459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:47.825762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:47.837957 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a12ce117-bf95-49c6-8616-076dd05a5d41 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.151990 May 23 16:21:47.838547 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2122/4136] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:47 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:48.853909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:48.854467 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:48.861082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:48.861538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:48.862671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:48.863149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:48.989638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:48.997730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-128fc478-3548-43bd-83db-b2433473d8ed tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.147750 May 23 16:21:48.998292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2015/4137] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:48 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:50.014161 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:50.014776 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:50.016740 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:50.017861 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=8 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:50.023271 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:50.023713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:50.024462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:50.024892 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:50.155945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:50.164234 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0eefb08d-64cd-4824-98c0-f00749c473ea tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.153819 May 23 16:21:50.164815 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2123/4138] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:50 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:50.744242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-f1ecc2f4-8f3b-4b6c-9dc1-0ace3e25105e req-bddd0f4c-2a04-413b-a5e0-73add9645982 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ad71e05-689f-42a2-8cf7-82c2a3562e9d", "name": "network-vif-plugged", "status": "completed", "tag": "8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:50.752386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-f1ecc2f4-8f3b-4b6c-9dc1-0ace3e25105e req-bddd0f4c-2a04-413b-a5e0-73add9645982 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:50.752840 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-f1ecc2f4-8f3b-4b6c-9dc1-0ace3e25105e req-bddd0f4c-2a04-413b-a5e0-73add9645982 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:50.765453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-f1ecc2f4-8f3b-4b6c-9dc1-0ace3e25105e req-bddd0f4c-2a04-413b-a5e0-73add9645982 service nova] Creating event network-vif-plugged:8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0 for instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d on hv-cinder-83588 May 23 16:21:50.770638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-f1ecc2f4-8f3b-4b6c-9dc1-0ace3e25105e req-bddd0f4c-2a04-413b-a5e0-73add9645982 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090261 May 23 16:21:50.774953 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2016/4139] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:21:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:51.180851 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:51.181468 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:51.187920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:51.188374 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:51.189135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:51.189577 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:51.193463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=99 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:51.316329 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:51.324399 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4de8d254-7d09-4c07-83f6-087905dc53dd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.147421 May 23 16:21:51.324960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2124/4140] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:51 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:52.338486 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=8 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:21:52.343155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:52.343870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:52.350727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:52.352241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:52.352241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:52.352466 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:52.483804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:52.491384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-58315788-f650-42db-a18b-f14ce6d3e509 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1362 microversion: 2.1 time: 0.153772 May 23 16:21:52.491940 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2017/4141] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:52 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:53.508199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:53.508844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:53.515275 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:53.515724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:53.516495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:53.516929 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:53.651630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:21:53.659508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8b1c5ae7-5d3f-4f1d-8545-770fefe61cc8 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1548 microversion: 2.1 time: 0.155531 May 23 16:21:53.660071 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2125/4142] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:21:53 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1548 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:21:53.675512 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:21:53.679206 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:21:53.682606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:53.683077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:53.683878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:53.684315 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:53.762548 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-fd1990bd-4a53-4224-a8ae-bde3b5993c94 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090784 May 23 16:21:53.763169 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2018/4143] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 16:21:53 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:21:54.995670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-775362c1-5091-4c8f-8199-5127ae349642 req-85af5ccf-7d60-427b-8352-21f3f73bd9b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ad71e05-689f-42a2-8cf7-82c2a3562e9d", "tag": "8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:21:55.004248 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-775362c1-5091-4c8f-8199-5127ae349642 req-85af5ccf-7d60-427b-8352-21f3f73bd9b9 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:21:55.004680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-775362c1-5091-4c8f-8199-5127ae349642 req-85af5ccf-7d60-427b-8352-21f3f73bd9b9 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:21:55.016315 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-775362c1-5091-4c8f-8199-5127ae349642 req-85af5ccf-7d60-427b-8352-21f3f73bd9b9 service nova] Creating event network-changed:8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0 for instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d on hv-cinder-83588 May 23 16:21:55.021763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-775362c1-5091-4c8f-8199-5127ae349642 req-85af5ccf-7d60-427b-8352-21f3f73bd9b9 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029778 May 23 16:21:55.022366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2126/4144] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:21:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:29.517317 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=45,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:29.518917 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=64 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:29.524388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cb2a9e65-ca08-4625-ae91-ac79a8dbd927"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:29.525901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:29.532861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:29.538394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:29.538394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:29.538394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:30.225147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=43,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:30.225874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=13 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:30.230703 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:30.231369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:30.237657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:30.238107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:30.238868 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:30.239360 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:30.262459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lazy-loading 'flavor' on Instance uuid 8920fd46-7732-48a4-a860-831350e77715 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:22:30.886808 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-92104218-d650-4517-b08a-cb42304cccf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/action" status: 202 len: 0 microversion: 2.20 time: 0.663204 May 23 16:22:30.887657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2127/4145] 10.222.0.32 () {62 vars in 1318 bytes} [Mon May 23 16:22:30 2022] POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/action => generated 0 bytes in 665 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 23 16:22:30.895884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:30.896623 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:30.903914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:30.904392 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:30.907294 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:30.907856 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:31.081940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:31.090533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:31.092144 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:31.093407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2b2967bc-9611-4f5f-a578-b6b75e8a4c45 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.201618 May 23 16:22:31.094016 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2128/4146] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:30 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:22:31.593114 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-30a2a46a-f36d-4c76-a671-ee739b6fa26e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.077744 May 23 16:22:31.598194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2019/4147] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 16:22:29 2022] POST /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments => generated 194 bytes in 2079 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:32.111084 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:32.111745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:32.118352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:32.118809 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:32.119620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:32.120057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:32.247382 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:32.255187 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:32.256694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:32.257877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-445421f1-a2c7-4d10-af95-5080c08a88cf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.150702 May 23 16:22:32.258417 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2129/4148] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:32 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:33.277754 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:33.278520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:33.285604 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:33.286629 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:33.287480 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:33.288309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:33.507427 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:33.516089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:33.517681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:33.518823 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ec8199d5-b674-4718-b749-d89ffcf0d6df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.246354 May 23 16:22:33.519434 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2020/4149] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:33 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:33.889515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-fbdcf2fa-4869-4d39-a0e1-d2f8ef927ce0 req-c9960978-2f3c-4233-8a84-cbfac3cdf4f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d808c02-6a3e-4e38-bfd1-986e31e8cbd2", "name": "network-vif-deleted", "tag": "16595806-91fa-49b4-9688-d528f91f3cca"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:33.898944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-fbdcf2fa-4869-4d39-a0e1-d2f8ef927ce0 req-c9960978-2f3c-4233-8a84-cbfac3cdf4f2 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:33.900407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-fbdcf2fa-4869-4d39-a0e1-d2f8ef927ce0 req-c9960978-2f3c-4233-8a84-cbfac3cdf4f2 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:33.913231 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-fbdcf2fa-4869-4d39-a0e1-d2f8ef927ce0 req-c9960978-2f3c-4233-8a84-cbfac3cdf4f2 service nova] Creating event network-vif-deleted:16595806-91fa-49b4-9688-d528f91f3cca for instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 on hv-cinder-83588 May 23 16:22:33.918506 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-fbdcf2fa-4869-4d39-a0e1-d2f8ef927ce0 req-c9960978-2f3c-4233-8a84-cbfac3cdf4f2 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033180 May 23 16:22:33.919660 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2130/4150] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:34.537112 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:34.537718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:34.544829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:34.545278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:34.546024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:34.546461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:34.672236 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:34.680035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:34.681526 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:34.682678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-0d3af365-ea60-4779-a4b5-1dac4bba3b2d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.149477 May 23 16:22:34.683253 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2021/4151] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:34 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:35.701347 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:35.702041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:35.708160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:35.708593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:35.709335 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:35.709762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:35.927984 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:35.936081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:35.937603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:35.938799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d8ef416a-3e94-486f-ba2c-8a1eb9c3b8df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.242144 May 23 16:22:35.939388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2131/4152] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:35 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:36.181009 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-045ba747-fc34-49b7-9a40-8e7b63c3d3b0 req-c364c0b2-cee1-464c-b57c-82c1eafdea91 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e090a96-ef41-4548-ac75-e4dd2f371539", "name": "network-vif-deleted", "tag": "bfba56dc-c508-4f1b-8805-42a00d24ea9e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:36.189479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-045ba747-fc34-49b7-9a40-8e7b63c3d3b0 req-c364c0b2-cee1-464c-b57c-82c1eafdea91 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:36.189930 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-045ba747-fc34-49b7-9a40-8e7b63c3d3b0 req-c364c0b2-cee1-464c-b57c-82c1eafdea91 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:36.201934 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-045ba747-fc34-49b7-9a40-8e7b63c3d3b0 req-c364c0b2-cee1-464c-b57c-82c1eafdea91 service nova] Creating event network-vif-deleted:bfba56dc-c508-4f1b-8805-42a00d24ea9e for instance 3e090a96-ef41-4548-ac75-e4dd2f371539 on hv-cinder-83588 May 23 16:22:36.207386 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-045ba747-fc34-49b7-9a40-8e7b63c3d3b0 req-c364c0b2-cee1-464c-b57c-82c1eafdea91 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030087 May 23 16:22:36.207980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2022/4153] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:36.956968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:36.957508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:36.964665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:36.965111 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:36.965867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:36.966309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:37.139567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:37.147418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:37.148919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:37.150102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-763daa21-4938-4aba-bb68-3dfcf35143f3 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.197126 May 23 16:22:37.152405 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2132/4154] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:36 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:38.175839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:38.182358 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:38.184639 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:38.185099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:38.185890 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:38.186331 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:38.222247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-7add635a-1039-45af-bd76-b1103b172767 req-0711cbca-1737-4700-9c63-2ba0c0b784a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "087911cb-6647-4769-a496-f0a0f2e0482c", "name": "network-vif-deleted", "tag": "7d2df762-e149-458d-aedd-407ed8492eb6"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:38.235644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-7add635a-1039-45af-bd76-b1103b172767 req-0711cbca-1737-4700-9c63-2ba0c0b784a7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:38.241345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-7add635a-1039-45af-bd76-b1103b172767 req-0711cbca-1737-4700-9c63-2ba0c0b784a7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:38.251394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-7add635a-1039-45af-bd76-b1103b172767 req-0711cbca-1737-4700-9c63-2ba0c0b784a7 service nova] Creating event network-vif-deleted:7d2df762-e149-458d-aedd-407ed8492eb6 for instance 087911cb-6647-4769-a496-f0a0f2e0482c on hv-cinder-83588 May 23 16:22:38.265015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-7add635a-1039-45af-bd76-b1103b172767 req-0711cbca-1737-4700-9c63-2ba0c0b784a7 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046609 May 23 16:22:38.267921 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2133/4155] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:22:38.363863 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:38.372648 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:38.374144 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:38.375320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e9dc19d2-dd35-4b66-8909-c7d672d8262b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.210358 May 23 16:22:38.375879 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2023/4156] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:38 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:39.394243 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:39.395608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:39.401900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:39.402353 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:39.403136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:39.403564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:39.532762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:39.540875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:39.543173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:39.544555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-075354eb-2027-45a9-b8a2-f0cee3e84e88 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.154987 May 23 16:22:39.545893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2134/4157] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:39 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:40.289773 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-b061be20-08d2-4e36-b376-4302ac0dbaf0 req-f2651531-3af3-45b2-a0a2-d1054e8db5ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a14c169-ee02-4abd-8bae-be450c73183c", "name": "network-vif-deleted", "tag": "c91045a8-7f3a-40f1-bb8a-d0e2283349f5"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:40.298649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-b061be20-08d2-4e36-b376-4302ac0dbaf0 req-f2651531-3af3-45b2-a0a2-d1054e8db5ce service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:40.299074 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-b061be20-08d2-4e36-b376-4302ac0dbaf0 req-f2651531-3af3-45b2-a0a2-d1054e8db5ce service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:40.313210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-b061be20-08d2-4e36-b376-4302ac0dbaf0 req-f2651531-3af3-45b2-a0a2-d1054e8db5ce service nova] Creating event network-vif-deleted:c91045a8-7f3a-40f1-bb8a-d0e2283349f5 for instance 3a14c169-ee02-4abd-8bae-be450c73183c on hv-cinder-83588 May 23 16:22:40.319598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-b061be20-08d2-4e36-b376-4302ac0dbaf0 req-f2651531-3af3-45b2-a0a2-d1054e8db5ce service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033376 May 23 16:22:40.320322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2024/4158] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:40.563206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:40.563862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:40.570388 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:40.570832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:40.572527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:40.572527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:40.706537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:40.715051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:40.716576 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:40.717801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e9766a46-8603-4fab-950e-a9231f5cba9f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.158470 May 23 16:22:40.718348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2135/4159] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:40 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:41.198911 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cb2a9e65-ca08-4625-ae91-ac79a8dbd927"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:41.200040 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:41.206643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.210708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.210708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.210708 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.287591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: POST /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments HTTP/1.1 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 74 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:22:41.289435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {"volumeAttachment": {"volumeId": "cb2a9e65-ca08-4625-ae91-ac79a8dbd927"}} May 23 16:22:41.297511 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'server_id': '64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd', 'body': {'volumeAttachment': {'volumeId': 'cb2a9e65-ca08-4625-ae91-ac79a8dbd927'}}} May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:41.298714 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:41.300419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 400 to user: Invalid volume: volume cb2a9e65-ca08-4625-ae91-ac79a8dbd927 is already attached to instances: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d454c5c4-20e3-491d-a949-d78726b39e9b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.097974 May 23 16:22:41.302964 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2025/4160] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 16:22:41 2022] POST /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments => generated 174 bytes in 99 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 23 16:22:41.353903 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:41.360001 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:41.361022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.361476 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.362274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.362714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.446196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lazy-loading 'services' on Instance uuid 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:22:41.685601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e5dace8d-079a-41c2-91df-91fa11b6434f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments/cb2a9e65-ca08-4625-ae91-ac79a8dbd927" status: 202 len: 0 microversion: 2.1 time: 0.335440 May 23 16:22:41.686383 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2136/4161] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:22:41 2022] DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments/cb2a9e65-ca08-4625-ae91-ac79a8dbd927 => generated 0 bytes in 337 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:22:41.694544 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:41.695542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:41.702262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.702718 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.703516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.703956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.733914 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8973d3eb-4d25-458c-95be-3391b4d2e610 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043360 May 23 16:22:41.734468 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2026/4162] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:22:41 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:41.737049 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:41.737864 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:41.744191 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.744644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.745403 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:41.745842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:41.877473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:41.885437 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:41.886982 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:41.888177 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-da3faf93-2b8a-431b-a5f1-8c2b6f8e0b81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.155565 May 23 16:22:41.888730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2137/4163] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:41 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:22:42.746219 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:42.747054 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:42.753596 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:42.754051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:42.754812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:42.755297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:42.784109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3ae0ae34-e4aa-4ac3-9584-626ae930aefb tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042550 May 23 16:22:42.784661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2027/4164] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:22:42 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:42.906667 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:42.907345 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:42.913755 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:42.914222 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:42.914983 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:42.915446 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:43.047015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:43.055385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:43.056879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:43.058692 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fd0e65eb-7122-4896-b4da-e1244eccd6fb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.156329 May 23 16:22:43.059274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2138/4165] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:42 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:43.795717 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:43.796589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:43.802956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:43.803479 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:43.804230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:43.804662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:43.835222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e2a9c681-baff-4e88-b451-4bcda9ff4593 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043457 May 23 16:22:43.835772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2028/4166] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:22:43 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 16:22:43.892368 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:43.892848 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:43.898751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:43.899237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:43.899989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:43.900418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:43.927151 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:22:44.077249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:44.077772 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:44.083725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:44.084178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:44.085002 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:44.085440 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:44.179863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6dd3ac4d-289e-45d4-b555-b44accf13645 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 204 len: 0 microversion: 2.1 time: 0.290917 May 23 16:22:44.180617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2139/4167] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:22:43 2022] DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:22:44.189721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:44.190391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:44.196744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:44.197192 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:44.197980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:44.199002 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:44.217226 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:44.220664 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:44.229100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:44.234975 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:44.236220 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-ad5882ea-f7ef-4cf2-8935-444f570543f7 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.163459 May 23 16:22:44.243401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2029/4168] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:44 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:22:44.335639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:44.343432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-79932f31-090e-4d47-b782-87df393f2cca tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1682 microversion: 2.1 time: 0.157873 May 23 16:22:44.343994 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2140/4169] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:44 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1682 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:22:45.254971 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:45.255621 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:45.262697 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:45.263184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:45.264029 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:45.264476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:45.361173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:45.361795 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:45.368715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:45.369171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:45.369976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:45.370416 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:45.432758 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:45.440724 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:45.442224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:45.443413 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4af074d5-a26d-4657-bf58-1f2067556e94 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.192382 May 23 16:22:45.443965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2030/4170] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:45 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:45.590894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:45.598635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0f4aecee-2d99-4102-bd8f-a9efca1de447 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1682 microversion: 2.1 time: 0.241358 May 23 16:22:45.599244 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2141/4171] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:45 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1682 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:22:46.383996 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-68be33f9-dc5a-484b-bb53-9b8b28506aae req-c044a313-a65d-4cad-8bc9-eb6caf953b81 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd", "name": "network-vif-deleted", "tag": "243e14c9-e626-4c45-8053-04ff3f1602dd"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:46.395692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-68be33f9-dc5a-484b-bb53-9b8b28506aae req-c044a313-a65d-4cad-8bc9-eb6caf953b81 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:46.401411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-68be33f9-dc5a-484b-bb53-9b8b28506aae req-c044a313-a65d-4cad-8bc9-eb6caf953b81 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:46.410195 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-68be33f9-dc5a-484b-bb53-9b8b28506aae req-c044a313-a65d-4cad-8bc9-eb6caf953b81 service nova] Creating event network-vif-deleted:243e14c9-e626-4c45-8053-04ff3f1602dd for instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd on hv-cinder-83588 May 23 16:22:46.415309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-68be33f9-dc5a-484b-bb53-9b8b28506aae req-c044a313-a65d-4cad-8bc9-eb6caf953b81 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034842 May 23 16:22:46.415915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2031/4172] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:46.462142 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:46.462757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:46.472245 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:46.472907 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:46.473944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:46.474460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:46.616573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:46.617612 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:46.626753 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:46.627565 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:46.628691 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:46.629369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:46.640695 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:46.650607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:46.655850 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:46.657862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf60de86-bf12-4643-941d-931269a40b57 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.199876 May 23 16:22:46.659395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2142/4173] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:46 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:46.708956 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:46.717713 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-84088038-9375-4e8c-9581-46d2a0b8850b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 200 len: 1326 microversion: 2.1 time: 0.105445 May 23 16:22:46.727155 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2032/4174] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:46 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 1326 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:22:47.676920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:47.677584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:47.684965 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:47.685409 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:47.686157 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:47.686590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:47.754205 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:47.754826 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:47.762066 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:47.762525 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:47.763322 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:47.763756 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:47.778303 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd HTTP/1.1 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:22:47.779986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:22:47.782137 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd'} May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:22:47.783165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:22:47.784821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:22:47.786280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:22:47.786280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4467b78b-5910-49fa-b5d8-c36094ae0d2b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 404 len: 111 microversion: 2.1 time: 0.032692 May 23 16:22:47.786280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2033/4175] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:47 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:22:47.790070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e172f887-12da-41d1-bec1-f515a0210c83 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '.version_select at 0x7f47f99ad5e0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:47.830829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:47.839675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:47.841170 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:47.842325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b9f20cbd-18db-4d77-a7de-9f653c5d43aa tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.169431 May 23 16:22:47.842882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2143/4176] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:47 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:47.878200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e172f887-12da-41d1-bec1-f515a0210c83 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-787462989" status: 202 len: 0 microversion: 2.1 time: 0.091598 May 23 16:22:47.878767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2034/4177] 10.222.0.32 () {58 vars in 1217 bytes} [Mon May 23 16:22:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-787462989 => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 23 16:22:48.630235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a2dc8fe4-f4ce-47a7-99d5-83f5a54778b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: .version_select at 0x7f47f9159b80>, body: {"keypair": {"name": "tempest-keypair-1293119602", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPRuDQQ5JnaSCBPrRfM7sHjmz+xUb2T/c38NqJsVxfI8kIqzBY2+rrIqTpSPAcU3ut89yS4VWwAdJ7JaYyKfXnCZuJFyHu24cMOzOqH8+hhsj4KECum2t+5+SnvEljhhiw=="}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:48.644917 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-a2dc8fe4-f4ce-47a7-99d5-83f5a54778b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:22:48.649428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.quota [None req-a2dc8fe4-f4ce-47a7-99d5-83f5a54778b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'key_pairs'} {{(pid=124966) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:22:48.729510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a2dc8fe4-f4ce-47a7-99d5-83f5a54778b1 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.103067 May 23 16:22:48.730266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2144/4178] 10.222.0.32 () {60 vars in 1158 bytes} [Mon May 23 16:22:48 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:48.860937 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:48.861553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:48.868311 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:48.868760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:48.869514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:48.869948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:49.018145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:49.027899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:49.028570 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:49.029764 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-72612f9b-3c1e-4ef9-ac58-5e58dc55c9b4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.173219 May 23 16:22:49.030324 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2035/4179] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:48 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:50.049588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:50.050207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:50.056827 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:50.057273 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:50.058341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:50.058779 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:50.531620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:50.539721 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:50.541299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:50.542542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-75be21f7-2654-4bca-b2db-2aad4da5d9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.497542 May 23 16:22:50.543215 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2145/4180] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:50 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:51.477357 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1677688755", "imageRef": "02252060-4166-40c5-b659-ae74a201ef83", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7e7a8587-a97e-4c11-9c4a-682adeb9c236"}], "security_groups": [{"name": "tempest-securitygroup--1162687605"}], "key_name": "tempest-keypair-1293119602", "user_data": "}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:51.560874 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:51.561430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:51.567923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:51.568373 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:51.569136 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:51.569739 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:51.578782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.network.neutron [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] validate_networks() for [('7e7a8587-a97e-4c11-9c4a-682adeb9c236', None, None, None, None, None)] {{(pid=124967) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 23 16:22:51.772543 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:51.781344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:51.782866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:51.784074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bb7bd747-112a-4f65-8b10-265049ec5531 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.227248 May 23 16:22:51.784635 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2146/4181] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:51 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:22:52.047380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 23 16:22:52.047844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image limits 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 23 16:22:52.048327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Flavor pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 23 16:22:52.048747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Image pref 0:0:0 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 23 16:22:52.049168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.virt.hardware [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124967) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 23 16:22:52.057924 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:52.058405 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:52.059246 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:52.059689 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:52.076665 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 23 16:22:52.080589 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.quota [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Getting quotas for user 72deeff329a6482cbfdb828a1996b127 and project f474f456861e40ee9eb654852f80f313. Resources: {'ram', 'instances', 'cores'} {{(pid=124967) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 23 16:22:52.092243 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Going to run 1 instances... {{(pid=124967) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 23 16:22:52.094501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='02252060-4166-40c5-b659-ae74a201ef83',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124967) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 23 16:22:52.233284 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a3a3713e-0b55-4705-b06f-d1d6c1f43f79 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.760134 May 23 16:22:52.234514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2036/4182] 10.222.0.32 () {60 vars in 1146 bytes} [Mon May 23 16:22:51 2022] POST /compute/v2.1/servers => generated 402 bytes in 762 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 23 16:22:52.244022 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-bcda0ad7-4161-491b-b4c8-64e58ea8943a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:52.245260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-bcda0ad7-4161-491b-b4c8-64e58ea8943a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:52.303397 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-bcda0ad7-4161-491b-b4c8-64e58ea8943a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:52.313953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-bcda0ad7-4161-491b-b4c8-64e58ea8943a tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1242 microversion: 2.1 time: 0.073723 May 23 16:22:52.314613 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2147/4183] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:52 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1242 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:52.802901 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:52.803707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:52.818302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:52.818302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:52.818302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:52.818302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:52.818696 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-f705b7ab-ba4b-40dc-909f-efe0e9ca6995 req-37b02ecb-8f61-4dc4-be61-e51c4ffb7d9f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "name": "network-vif-plugged", "status": "completed", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:52.828297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f705b7ab-ba4b-40dc-909f-efe0e9ca6995 req-37b02ecb-8f61-4dc4-be61-e51c4ffb7d9f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:52.828919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-f705b7ab-ba4b-40dc-909f-efe0e9ca6995 req-37b02ecb-8f61-4dc4-be61-e51c4ffb7d9f service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:52.840775 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-f705b7ab-ba4b-40dc-909f-efe0e9ca6995 req-37b02ecb-8f61-4dc4-be61-e51c4ffb7d9f service nova] Creating event network-vif-plugged:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:22:52.845149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-f705b7ab-ba4b-40dc-909f-efe0e9ca6995 req-37b02ecb-8f61-4dc4-be61-e51c4ffb7d9f service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033570 May 23 16:22:52.845976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2148/4184] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:52.956903 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:52.966925 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:52.967680 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:52.968935 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8b17127e-ab95-4c41-8b1f-c9bc75608f81 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.170923 May 23 16:22:52.969815 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2037/4185] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:52 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:53.332309 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:53.333012 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:53.339879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:53.340327 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:53.341085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:53.341524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:53.405612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:53.413264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-366b36b7-3220-43e6-806b-e56fb491bc9f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1246 microversion: 2.1 time: 0.085456 May 23 16:22:53.413832 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2149/4186] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:53 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1246 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:53.989686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:53.990307 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:53.997290 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:53.997749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:53.998516 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:53.998954 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:54.017244 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:22:54.132701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:54.141092 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:54.142603 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:54.143809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-60ed4a59-6cc1-4660-95e8-ca829bf777ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.158115 May 23 16:22:54.144401 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2038/4187] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:53 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:54.429877 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:54.430885 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:54.438263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:54.438263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:54.438263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:54.438646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:54.513745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:54.522249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-110d9a88-48b9-4e87-967b-0df2579df421 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1312 microversion: 2.1 time: 0.096328 May 23 16:22:54.522838 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2150/4188] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:54 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1312 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:55.163365 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:55.167280 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:55.171531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:55.172103 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:55.172989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:55.173678 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:55.430673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:55.439709 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:55.441346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:55.442643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-77707171-b2e0-4871-8c03-8e05e346d30b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1742 microversion: 2.20 time: 0.283388 May 23 16:22:55.443335 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2039/4189] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:55 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1742 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:55.552747 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:55.553816 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:55.562146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:55.563253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:55.564542 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:55.565484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:55.761582 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:55.769109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ff706a1b-abe9-4f62-9a79-3eea9df73b48 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.220989 May 23 16:22:55.769932 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2151/4190] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:55 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:56.273319 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-5d38a48a-1164-43bc-a21d-884586ba6ec3 req-3bf41316-e56a-4d02-9652-5e2f6f2e6ecb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53650270-86f7-485b-931c-754469e60288", "tag": "9860198a-b597-499a-b2ce-fe716df5978e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:22:56.297394 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5d38a48a-1164-43bc-a21d-884586ba6ec3 req-3bf41316-e56a-4d02-9652-5e2f6f2e6ecb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:56.298429 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5d38a48a-1164-43bc-a21d-884586ba6ec3 req-3bf41316-e56a-4d02-9652-5e2f6f2e6ecb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:56.327387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-5d38a48a-1164-43bc-a21d-884586ba6ec3 req-3bf41316-e56a-4d02-9652-5e2f6f2e6ecb service nova] Creating event network-changed:9860198a-b597-499a-b2ce-fe716df5978e for instance 53650270-86f7-485b-931c-754469e60288 on hv-cinder-83588 May 23 16:22:56.338681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-5d38a48a-1164-43bc-a21d-884586ba6ec3 req-3bf41316-e56a-4d02-9652-5e2f6f2e6ecb service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076489 May 23 16:22:56.340751 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2040/4191] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:22:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:22:56.461146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:56.461675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:56.471806 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:56.475840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:56.476902 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:56.477607 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:56.737865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:56.748899 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:56.753439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:56.754896 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e26da4fc-7d36-4372-873c-c12482456913 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.297743 May 23 16:22:56.755968 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2152/4192] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:56 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:56.786958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:56.788037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:56.797439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:56.798260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:56.799188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:56.799679 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:57.029493 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:57.045175 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2d4be961-02bb-40c0-b19a-a4ec71e8869d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.262461 May 23 16:22:57.054325 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2041/4193] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:56 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:57.773676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:57.774276 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:57.781895 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:57.782344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:57.783127 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:57.783564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:57.947544 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:57.962394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:57.964922 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:57.966113 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-398a902e-f708-4f80-b2b0-b2965f356787 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.196328 May 23 16:22:57.966663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2153/4194] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:57 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:58.067531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:58.068656 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:58.076399 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:58.077729 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:58.079278 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:58.079747 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:58.221056 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:58.229005 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-686b360a-f2ca-41b7-8203-3a8f7f0c5c8b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.170378 May 23 16:22:58.231271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2042/4195] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:58 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:22:58.984867 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:58.985487 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:58.992074 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:58.992527 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:58.993286 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:58.993723 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:59.127089 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:59.136097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:59.137595 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:59.138967 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-40a8a973-f4e4-4efc-9608-d10364b34faf tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.158072 May 23 16:22:59.139875 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2154/4196] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:22:58 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:22:59.247125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:22:59.248034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:22:59.254533 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:59.255261 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:59.256274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:22:59.256683 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:22:59.415188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:22:59.422993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-633d939f-c143-442d-b902-13cacc755519 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.179727 May 23 16:22:59.432423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2043/4197] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:22:59 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:00.158249 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:00.159602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:00.167706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:00.168452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:00.169439 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:00.170106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:00.312753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:00.322193 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:00.324810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:00.326264 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5aaa49a9-d655-4c8a-b3bd-fbe9c31123df tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.172451 May 23 16:23:00.327023 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2155/4198] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:00 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:00.439785 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:00.441057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:00.443740 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:00.452122 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:00.454581 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:00.455757 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:00.456609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:00.609057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:00.617250 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b78c6374-9173-4fda-b9dd-726d912a4886 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.181185 May 23 16:23:00.617821 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2044/4199] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:00 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:01.344810 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:01.346260 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:01.348909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=75,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:01.354146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:01.354919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:01.356125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:01.356631 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:01.505811 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:01.515890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:01.517676 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:01.518915 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6712ff38-629b-4a10-b851-96fdfd8ac3db tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.177998 May 23 16:23:01.519515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2156/4200] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:01 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:01.633782 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:01.634439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:01.641781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:01.642181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:01.643196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:01.643817 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:01.789059 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:01.799182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e3130365-1109-43b2-9060-496aa7741443 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.169277 May 23 16:23:01.801907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2045/4201] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:01 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 23 16:23:02.537900 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:02.538515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:02.540465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell0:SELECT=6 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:02.546171 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:02.546620 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:02.547402 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:02.547841 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:02.682981 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:02.691608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:02.693186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:02.694486 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d6dfe511-01c0-4559-80b9-ff4152e894ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.160507 May 23 16:23:02.695541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2157/4202] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:02 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:02.820989 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:02.821611 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:02.825576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:02.830411 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell0:SELECT=6 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:02.839356 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:02.840256 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:02.841126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:02.841643 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:02.998640 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:03.007314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d7d584aa-8c44-44df-a25c-ef6d40881b4f tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.190294 May 23 16:23:03.017703 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2046/4203] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:02 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:03.713394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:03.714038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:03.723657 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:03.724028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:03.725724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:03.725724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:03.871677 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:03.882817 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:03.884460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:03.885801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e69e4b28-e4c8-4eb7-82aa-f318ada0f541 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.176449 May 23 16:23:03.886540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2158/4204] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:03 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:04.024845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:04.025465 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:04.032803 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:04.033262 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:04.034037 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:04.034471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:04.219880 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:04.228414 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8dcc7c06-1ee8-4d60-9293-13523d412b33 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.207289 May 23 16:23:04.233109 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2047/4205] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:04 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:04.906412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:04.907150 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:04.917987 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:04.918618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:04.919520 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:04.920097 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:05.039341 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-d5d73234-5938-4563-b149-b0b24af72d22 req-f9864787-60c8-4c3b-be47-3ab4396669f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53650270-86f7-485b-931c-754469e60288", "name": "network-vif-plugged", "status": "completed", "tag": "9860198a-b597-499a-b2ce-fe716df5978e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:05.053435 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d5d73234-5938-4563-b149-b0b24af72d22 req-f9864787-60c8-4c3b-be47-3ab4396669f7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:05.054135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-d5d73234-5938-4563-b149-b0b24af72d22 req-f9864787-60c8-4c3b-be47-3ab4396669f7 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:05.070164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-d5d73234-5938-4563-b149-b0b24af72d22 req-f9864787-60c8-4c3b-be47-3ab4396669f7 service nova] Creating event network-vif-plugged:9860198a-b597-499a-b2ce-fe716df5978e for instance 53650270-86f7-485b-931c-754469e60288 on hv-cinder-83588 May 23 16:23:05.077115 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-d5d73234-5938-4563-b149-b0b24af72d22 req-f9864787-60c8-4c3b-be47-3ab4396669f7 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042702 May 23 16:23:05.078316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2048/4206] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:23:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:23:05.094665 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:05.100819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=100 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:05.106320 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:05.108018 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:05.109263 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-84619d91-259b-4c53-a1ad-45635910365a tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.208176 May 23 16:23:05.110115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2159/4207] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:04 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:05.253419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:05.254141 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:05.266096 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:05.266549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:05.267372 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:05.268046 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:05.407502 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:05.414995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3351e963-6d94-4026-ad4b-dd93d9c30191 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.165910 May 23 16:23:05.415606 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2049/4208] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:05 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:06.129065 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:06.129694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:06.136757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:06.137194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:06.137960 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:06.138391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:06.280611 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:06.290537 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:06.292280 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:06.293529 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f8652ac3-790e-4c55-a8bc-8c214e8726ac tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.168535 May 23 16:23:06.294104 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2160/4209] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:06 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:06.433017 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:06.433617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:06.440532 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:06.440978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:06.441761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:06.442196 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:06.570129 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:06.578067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8ba6b29a-3919-4302-a5cf-f2ce418d840b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.148987 May 23 16:23:06.578624 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2050/4210] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:06 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:07.312350 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:07.312894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:07.319407 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:07.319863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:07.320597 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:07.321024 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:07.450015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:07.457829 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:07.459346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:07.460496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-882ca63c-b138-4fe8-a8ac-6d323ef8cace tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1737 microversion: 2.20 time: 0.152125 May 23 16:23:07.461051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2161/4211] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:07 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:07.595221 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:07.595781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:07.602332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:07.602774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:07.603546 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:07.603984 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:07.737846 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:07.745958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-9452c68b-83dd-40ee-8386-c7ed2ff49e57 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1368 microversion: 2.1 time: 0.154351 May 23 16:23:07.746524 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2051/4212] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:07 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1368 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:08.478438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:08.479045 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:08.485524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:08.486029 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:08.486730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:08.487180 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:08.697941 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:08.706694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:08.708295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:08.709481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e64ab0f2-2895-4ca2-8a08-13cdd4c8c348 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1719 microversion: 2.20 time: 0.234997 May 23 16:23:08.710295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2162/4213] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:08 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1719 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:08.763663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:08.765342 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:08.771811 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:08.772269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:08.773053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:08.773508 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:08.969591 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:08.977100 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-502110e7-d5fa-4e30-bd9d-99302655cf54 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1558 microversion: 2.1 time: 0.218236 May 23 16:23:08.977657 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2052/4214] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:23:08 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1558 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:23:08.994349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:08.994972 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:09.001528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:09.001980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:09.002736 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:09.003197 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:09.099008 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-9abd3c23-6573-4141-ba27-98688119b189 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.108490 May 23 16:23:09.099644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2163/4215] 10.222.0.32 () {58 vars in 1274 bytes} [Mon May 23 16:23:08 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-interface => generated 284 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:09.727471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:09.728089 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:09.734582 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:09.735039 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:09.735833 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:09.736279 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:10.132861 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:10.142192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:10.143725 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:10.144894 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dfba88c8-05c7-489e-936f-307f01f9e2e0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1719 microversion: 2.20 time: 0.421417 May 23 16:23:10.145455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2053/4216] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:09 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1719 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:10.331798 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-18b01142-a6cf-49ae-9522-075da08d400c req-478cba5b-c95f-432e-9f38-2c7d8ce7383a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "53650270-86f7-485b-931c-754469e60288", "tag": "9860198a-b597-499a-b2ce-fe716df5978e"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:10.339986 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-18b01142-a6cf-49ae-9522-075da08d400c req-478cba5b-c95f-432e-9f38-2c7d8ce7383a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:10.340507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-18b01142-a6cf-49ae-9522-075da08d400c req-478cba5b-c95f-432e-9f38-2c7d8ce7383a service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:10.351890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-18b01142-a6cf-49ae-9522-075da08d400c req-478cba5b-c95f-432e-9f38-2c7d8ce7383a service nova] Creating event network-changed:9860198a-b597-499a-b2ce-fe716df5978e for instance 53650270-86f7-485b-931c-754469e60288 on hv-cinder-83588 May 23 16:23:10.357332 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-18b01142-a6cf-49ae-9522-075da08d400c req-478cba5b-c95f-432e-9f38-2c7d8ce7383a service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029767 May 23 16:23:10.358007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2164/4217] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:23:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:23:11.030854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-f28d2e72-0d39-482b-90a1-6e43bcfd861f req-034cc788-befc-4c02-95e2-1a9a37f96c20 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "name": "network-vif-unplugged", "status": "completed", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:11.048281 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-f28d2e72-0d39-482b-90a1-6e43bcfd861f req-034cc788-befc-4c02-95e2-1a9a37f96c20 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:11.048731 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-f28d2e72-0d39-482b-90a1-6e43bcfd861f req-034cc788-befc-4c02-95e2-1a9a37f96c20 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:11.060993 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.compute.server_external_events [req-f28d2e72-0d39-482b-90a1-6e43bcfd861f req-034cc788-befc-4c02-95e2-1a9a37f96c20 service nova] Unable to find a host for instance 8920fd46-7732-48a4-a860-831350e77715. Dropping event network-vif-unplugged {{(pid=124967) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} May 23 16:23:11.062188 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-f28d2e72-0d39-482b-90a1-6e43bcfd861f req-034cc788-befc-4c02-95e2-1a9a37f96c20 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.035330 May 23 16:23:11.062907 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2054/4218] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:23:11 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 37 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) May 23 16:23:11.165015 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:11.165652 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:11.172579 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:11.173038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:11.174094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:11.174535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:11.313601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:11.322641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:11.324980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:11.326346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7bed6b45-6655-46ff-88cb-fa27a6006297 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1662 microversion: 2.20 time: 0.165745 May 23 16:23:11.326909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2165/4219] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:11 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1662 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:11.345162 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4c26b05d-0218-4f9a-9df7-25859b0f5a73"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:11.346202 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:11.353681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:11.353681 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:11.353976 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:11.354426 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:12.103057 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:12.476373 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5087f4c9-b741-48bb-b338-a91452b33143 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.135400 May 23 16:23:12.477007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2055/4220] 10.222.0.32 () {62 vars in 1363 bytes} [Mon May 23 16:23:11 2022] POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments => generated 188 bytes in 1137 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 23 16:23:12.594664 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:12.595202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:12.601734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:12.602181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:12.602926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:12.603465 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:13.236356 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-0278d26d-0b85-4d72-a8cd-a6cf4efd15c0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments/4c26b05d-0218-4f9a-9df7-25859b0f5a73" status: 202 len: 0 microversion: 2.20 time: 0.645610 May 23 16:23:13.237265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2166/4221] 10.222.0.32 () {60 vars in 1456 bytes} [Mon May 23 16:23:12 2022] DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments/4c26b05d-0218-4f9a-9df7-25859b0f5a73 => generated 0 bytes in 648 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 16:23:13.292645 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:13.293225 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:13.300451 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:13.300910 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:13.301676 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:13.302125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:13.556007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-64c67ec2-08da-4096-bc6a-d0e9b87d6914 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/action" status: 202 len: 0 microversion: 2.20 time: 0.267128 May 23 16:23:13.556889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2056/4222] 10.222.0.32 () {62 vars in 1318 bytes} [Mon May 23 16:23:13 2022] POST /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/action => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) May 23 16:23:13.565293 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:13.566106 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:13.572434 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:13.572882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:13.573639 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:13.574072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:13.708059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:13.715871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:13.717371 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:13.718535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a168dec4-0ff5-46c9-aa59-fb8587c399d6 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1670 microversion: 2.20 time: 0.157252 May 23 16:23:13.719110 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2167/4223] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:13 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1670 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:14.737393 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:14.738053 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:14.744126 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:14.744578 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:14.745338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:14.745774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:14.877238 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:14.885227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:14.887398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:14.888520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-6d62067d-8aa2-4552-b537-b8f612b93ca9 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1670 microversion: 2.20 time: 0.155605 May 23 16:23:14.889077 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2057/4224] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:14 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1670 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:15.906546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:15.907793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:15.914156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:15.914608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:15.915387 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:15.915824 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:16.112727 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:16.120949 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:16.122473 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:16.123682 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-1b85590a-4146-4a27-bec1-49cab6ffe031 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.221894 May 23 16:23:16.124253 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2168/4225] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:15 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:17.142912 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:17.143788 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:17.151216 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:17.151760 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:17.153138 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:17.153677 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:17.311675 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:17.320060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:17.321688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:17.322945 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dd5b312a-5b4f-490e-854c-79f39a01e0a2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.184800 May 23 16:23:17.334576 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2058/4226] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:17 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:18.341414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:18.342094 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:18.348659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:18.349116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:18.349890 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:18.350328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:18.508746 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:18.516602 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:18.518088 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:18.519274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-84c272fa-8f72-4c61-b6aa-82c1c99b7a5f tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.182301 May 23 16:23:18.519831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2169/4227] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:18 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:19.538218 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:19.538866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:19.546692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:19.547305 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:19.548150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:19.548710 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:19.694500 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:19.703514 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:19.705034 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:19.706213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89e77fdd-ef72-45a4-b2ed-022e42a853d0 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.171911 May 23 16:23:19.706949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2059/4228] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:19 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:20.724950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:20.725568 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:20.732393 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:20.737974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:20.737974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:20.737974 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:20.869535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:20.878217 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:20.879842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:20.881034 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2fbb2048-a622-4ccf-ada1-1620ddbe9c1d tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.159991 May 23 16:23:20.881715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2170/4229] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:20 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:21.670124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:21.671338 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:21.678970 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:21.679464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:21.680245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:21.680686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:21.703487 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-49f783ec-10dd-4814-b3ea-baa9b3a09a63 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/ips" status: 200 len: 146 microversion: 2.1 time: 0.038090 May 23 16:23:21.704051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2060/4230] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 16:23:21 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/ips => generated 146 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:21.899381 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:21.899980 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:21.906675 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:21.907145 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:21.907923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:21.908443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:22.049898 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:22.053762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:22.055617 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:22.057058 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cbc803fa-dec1-455a-8e2f-47c3e1c71517 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.161494 May 23 16:23:22.057541 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2171/4231] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:21 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:23.076035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:23.076642 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:23.083165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:23.083649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:23.084409 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:23.084860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:23.220905 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:23.228787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:23.230332 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:23.232082 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-b105c40a-613f-4bfc-8a0f-48e63d4ae82b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.159942 May 23 16:23:23.232632 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2061/4232] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:23 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:24.250603 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:24.256129 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:24.258066 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:24.258522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:24.259325 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:24.259772 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:24.393662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:24.401909 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:24.404313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:24.405823 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d13de773-55ae-4416-8bdf-9aba76b31762 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.159876 May 23 16:23:24.406713 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2172/4233] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:24 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:24.470672 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6bb179c4-1747-4821-b44e-2d29883e2f08"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:24.471858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:24.478701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:24.479198 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:24.479973 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:24.480412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:25.425336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:25.425973 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:25.432425 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:25.432870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:25.433633 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:25.434566 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:25.569395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:25.578078 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:25.579621 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:25.580783 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-5076067e-3345-4a04-b8f5-bab7e62d9a6b tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.159465 May 23 16:23:25.581340 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2173/4234] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:25 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:26.599598 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:26.600442 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:26.607627 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:26.608174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:26.610441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:26.610441 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:26.750669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:26.758546 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:26.760085 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:26.761262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-610c13f4-dc00-4266-8439-bb0a6190b45e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.165536 May 23 16:23:26.761925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2174/4235] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:26 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:27.779042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:27.779819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:27.786993 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:27.787480 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:27.788241 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:27.788670 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:27.791453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:27.950724 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:27.959429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:27.960945 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:27.962278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-76e2ce09-4327-4553-81ed-1f4988f1f951 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.187120 May 23 16:23:27.962870 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2175/4236] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:27 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:28.981573 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:28.982238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:28.988636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:28.989223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:28.989866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:28.990346 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:29.128228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:29.130936 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:29.132477 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:29.133636 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8342f45f-8339-4cae-b119-e1a150e1f9f8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.155969 May 23 16:23:29.134265 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2176/4237] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:28 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:30.151977 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:30.153228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:30.159863 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:30.160306 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:30.161055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:30.161489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:30.288728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:30.296654 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:30.298149 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:30.299352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-209b4af6-f1b9-46b7-8bda-b838fe608691 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.151206 May 23 16:23:30.299911 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2177/4238] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:30 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:31.317638 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:31.318268 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:31.325252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:31.325698 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:31.326453 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:31.326888 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:31.451641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:31.459459 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:31.460969 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:31.462120 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8227cf61-0eb1-4503-9b10-b216b11a7ab4 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.148470 May 23 16:23:31.462711 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2178/4239] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:31 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:32.480578 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:32.481223 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:32.483495 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=46 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:32.488199 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:32.488646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:32.489404 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:32.489839 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:32.622954 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:32.630940 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:32.632510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:32.633674 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c611caf1-6fef-4f1e-866e-4724e1089318 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.157213 May 23 16:23:32.634228 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2179/4240] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:32 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:33.652535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:33.653256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:33.660612 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:33.661070 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:33.661862 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:33.662299 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:33.795321 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:33.803884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:33.805413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:33.806580 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-03a0caea-e0d7-4dd9-8753-2f8bdf993ab8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.158079 May 23 16:23:33.807198 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2180/4241] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:33 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:34.476242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=39 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:34.582767 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=48,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:34.825348 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:34.825950 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:34.832554 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:34.833007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:34.833763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:34.834196 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:34.962944 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:34.971378 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:34.973925 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:34.975090 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-581f20f9-64c0-437a-a60a-ebcea3098243 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.153857 May 23 16:23:34.975686 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2181/4242] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:34 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 23 16:23:35.993452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:35.997842 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:36.002359 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:36.002819 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:36.003615 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:36.004055 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:36.213691 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:36.222105 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:36.223629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:36.224787 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce2b9eaf-1d0d-4474-861a-2e5e8aaecc55 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.235501 May 23 16:23:36.225344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2182/4243] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:35 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:36.381520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d2b60a3f-e578-4a84-9598-bc47408197bd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 11.914630 May 23 16:23:36.386778 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2062/4244] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 16:23:24 2022] POST /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 194 bytes in 11921 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:37.243313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:37.244530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:37.251884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:37.252341 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:37.253116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:37.253569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:37.384680 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:37.393616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:37.395186 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:37.396357 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-4cd70f3a-0813-43a8-9de4-8ea409cc4720 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.157257 May 23 16:23:37.396916 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2183/4245] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:37 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:38.416538 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:38.417242 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:38.423495 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:38.423950 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:38.424707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:38.425142 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:38.556422 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:38.570594 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:38.572158 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:38.573299 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e418e7a4-4783-4ff1-9511-655817fb8978 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.161469 May 23 16:23:38.573858 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2063/4246] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:38 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:39.076535 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-9e44a5f1-8c58-4005-98ab-f73b5ef3dce4 req-7b8a338c-4727-4b23-b3a1-7574d015f8da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "name": "network-vif-plugged", "status": "completed", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:39.085618 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-9e44a5f1-8c58-4005-98ab-f73b5ef3dce4 req-7b8a338c-4727-4b23-b3a1-7574d015f8da service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:39.086072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-9e44a5f1-8c58-4005-98ab-f73b5ef3dce4 req-7b8a338c-4727-4b23-b3a1-7574d015f8da service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:39.096786 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-9e44a5f1-8c58-4005-98ab-f73b5ef3dce4 req-7b8a338c-4727-4b23-b3a1-7574d015f8da service nova] Creating event network-vif-plugged:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:23:39.101831 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-9e44a5f1-8c58-4005-98ab-f73b5ef3dce4 req-7b8a338c-4727-4b23-b3a1-7574d015f8da service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029048 May 23 16:23:39.102481 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2184/4247] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:23:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:39.597688 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:39.598310 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:39.604779 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:39.605241 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:39.606008 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:39.606449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:39.743213 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:39.751721 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:39.753217 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:39.754370 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-588f0eb4-0932-403a-9b7f-1223130c651e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.161092 May 23 16:23:39.754929 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2064/4248] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:39 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:40.773255 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:40.773955 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:40.780096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:40.780571 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:40.782122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:40.782559 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:40.909262 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:40.917372 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:40.919068 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:40.921846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b8ccd025-c9f7-4a61-8d80-41bd2d956393 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.151712 May 23 16:23:40.921846 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2185/4249] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:40 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:41.938774 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:41.939420 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:41.946049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:41.946504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:41.947314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:41.947845 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:42.078792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:42.086652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:42.088180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:42.089390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7b80cb31-c671-44b6-9938-b09778ff1eca tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1728 microversion: 2.20 time: 0.154776 May 23 16:23:42.089946 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2065/4250] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:41 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1728 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:43.108471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:43.109114 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:43.116391 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:43.116840 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:43.117593 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:43.118028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:43.253463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:43.261428 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:43.262920 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:23:43.264107 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-7eb832c1-0a2e-45e5-a4db-f131192e28ec tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1715 microversion: 2.20 time: 0.159589 May 23 16:23:43.264717 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2186/4251] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:23:43 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1715 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:23:44.932024 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:44.932958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:44.939441 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:44.939978 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:44.940746 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:44.941182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:44.969958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8c7035fd-9972-4319-890f-794ea5253fd5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041823 May 23 16:23:44.970505 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2066/4252] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:23:44 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:44.981162 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:44.981663 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:44.988502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:44.988953 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:44.989730 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:44.990168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:45.018715 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-dfda3e5f-f203-4bf2-ba18-d739f29be5db tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08" status: 200 len: 194 microversion: 2.1 time: 0.041077 May 23 16:23:45.019317 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2187/4253] 10.222.0.32 () {58 vars in 1412 bytes} [Mon May 23 16:23:44 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:47.661320 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "122b3a7a-8d77-4712-aeb9-ef07a6ffea4a"}} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:23:47.662398 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:47.669083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:47.669534 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:47.670327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:47.670763 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:48.443896 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-db8c0e18-f75c-41bd-8811-6ad4686e7f7b tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "POST /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.787187 May 23 16:23:48.444542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2067/4254] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 16:23:47 2022] POST /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 194 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:56.980181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:56.981115 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:56.983753 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=17 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:56.984538 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=73 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:23:56.989038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:56.989489 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:56.990242 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:56.990673 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:57.020043 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-23b9cd1f-486e-4d33-9307-dd94174a412a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.043195 May 23 16:23:57.020590 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2188/4255] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:23:56 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 371 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:57.165504 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:57.166184 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:57.172598 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:57.173068 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:57.173825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:57.174254 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:57.202388 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-eeabd46c-c2c1-4a47-a871-55d265995562 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08" status: 200 len: 194 microversion: 2.1 time: 0.043323 May 23 16:23:57.203131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2068/4256] 10.222.0.32 () {58 vars in 1412 bytes} [Mon May 23 16:23:57 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:57.214254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:57.214748 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:57.221316 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:57.221757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:57.222510 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:57.223562 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:57.305886 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 1ad71e05-689f-42a2-8cf7-82c2a3562e9d {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:23:57.488054 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3ca0fb0f-585e-477f-9ceb-8901be323ec9 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08" status: 202 len: 0 microversion: 2.1 time: 0.278224 May 23 16:23:57.488763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2189/4257] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:23:57 2022] DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/6bb179c4-1747-4821-b44e-2d29883e2f08 => generated 0 bytes in 280 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:23:59.650555 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:59.651070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:59.657799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:59.658245 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:59.658998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:59.659471 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:59.688087 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d83869e2-a04c-45a6-b342-ea712c6feca5 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/122b3a7a-8d77-4712-aeb9-ef07a6ffea4a" status: 200 len: 194 microversion: 2.1 time: 0.041533 May 23 16:23:59.688653 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2069/4258] 10.222.0.32 () {58 vars in 1412 bytes} [Mon May 23 16:23:59 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/122b3a7a-8d77-4712-aeb9-ef07a6ffea4a => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:23:59.699168 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:23:59.699758 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:23:59.705963 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:59.706413 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:59.707189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:23:59.707629 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:23:59.786460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'services' on Instance uuid 1ad71e05-689f-42a2-8cf7-82c2a3562e9d {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:24:00.007366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-3d17581f-b11f-4005-b2e8-2f1b6c881efd tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/122b3a7a-8d77-4712-aeb9-ef07a6ffea4a" status: 202 len: 0 microversion: 2.1 time: 0.312711 May 23 16:24:00.007962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2190/4259] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:23:59 2022] DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments/122b3a7a-8d77-4712-aeb9-ef07a6ffea4a => generated 0 bytes in 314 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:24:02.213212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:02.214210 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:02.220661 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.221117 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.221875 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.222316 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.251200 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1afd7bd4-7feb-4997-bb12-c53403160936 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041887 May 23 16:24:02.251762 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2070/4260] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:24:02 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 16:24:02.355370 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:02.356438 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:02.362585 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.363033 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.364375 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.364818 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.392844 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-046c82c8-fd6c-42a6-8659-239cbebefa35 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042175 May 23 16:24:02.393304 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2191/4261] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:24:02 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 16:24:02.449844 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:02.450337 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:02.456363 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.456813 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.458069 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.458501 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.482662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:24:02.759464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-5da9a332-a06a-4f64-b22c-0a0e343a5a2e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 204 len: 0 microversion: 2.1 time: 0.313010 May 23 16:24:02.760086 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2071/4262] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:02 2022] DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:24:02.771420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:02.772530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:02.780208 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.780658 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.781418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:02.781849 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:02.908912 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:02.916274 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ca7cfaa7-503e-4477-b95f-126692e40111 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1673 microversion: 2.1 time: 0.151793 May 23 16:24:02.916834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2192/4263] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:02 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1673 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:03.933804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:03.934506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:03.941030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:03.941518 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:03.942292 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:03.942727 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:04.075249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:04.083364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-bca098ab-a9b1-42ed-b1d5-31168131634a tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1673 microversion: 2.1 time: 0.153804 May 23 16:24:04.083928 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2072/4264] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:03 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1673 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:05.100933 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:05.101471 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:05.107991 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:05.108445 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:05.109266 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:05.109706 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:05.343233 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:05.355533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-15fa973f-2919-4e0a-a41a-82fc9560963f tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1673 microversion: 2.1 time: 0.255121 May 23 16:24:05.355533 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2193/4265] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:05 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1673 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:06.047899 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-5e0bd8f4-655c-40be-95ef-7c1f6360d628 req-0d263f2f-790d-4f8b-b4ba-6eb6a32cbdd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ad71e05-689f-42a2-8cf7-82c2a3562e9d", "name": "network-vif-deleted", "tag": "8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:24:06.052463 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:06.058010 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5e0bd8f4-655c-40be-95ef-7c1f6360d628 req-0d263f2f-790d-4f8b-b4ba-6eb6a32cbdd0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:06.058453 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-5e0bd8f4-655c-40be-95ef-7c1f6360d628 req-0d263f2f-790d-4f8b-b4ba-6eb6a32cbdd0 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:06.062149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:06.070507 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-5e0bd8f4-655c-40be-95ef-7c1f6360d628 req-0d263f2f-790d-4f8b-b4ba-6eb6a32cbdd0 service nova] Creating event network-vif-deleted:8d520447-fea9-4ab9-8f3a-1acb3a2e2cd0 for instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d on hv-cinder-83588 May 23 16:24:06.075671 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-5e0bd8f4-655c-40be-95ef-7c1f6360d628 req-0d263f2f-790d-4f8b-b4ba-6eb6a32cbdd0 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031646 May 23 16:24:06.076271 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2073/4266] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:24:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:06.369095 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:06.371041 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:06.377952 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:06.380801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:06.380801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:06.380801 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:06.411601 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:06.412148 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:06.412608 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lazy-loading 'fault' on Instance uuid 1ad71e05-689f-42a2-8cf7-82c2a3562e9d {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:24:06.452910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8b0a82159c6343d28546ec7c775b8791', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e80aa9835e9845e798b1c8346de141e3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:06.460427 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fdeb77f8-1b74-4c48-91de-ac3dec7f4880 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 200 len: 1327 microversion: 2.1 time: 0.095404 May 23 16:24:06.460989 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2194/4267] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:06 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 1327 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:07.478455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:07.479070 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:07.487183 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:07.487628 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:07.488384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:07.488812 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:07.501734 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d HTTP/1.1 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:07.503384 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:07.505430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '1ad71e05-689f-42a2-8cf7-82c2a3562e9d'} May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:07.506430 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:07.507982 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:07.509637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 404 to user: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:07.509637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-250ebe44-a1bc-4c72-af06-a77c500b24a1 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 404 len: 111 microversion: 2.1 time: 0.030546 May 23 16:24:07.509637 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2074/4268] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:07 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:07.513482 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-e1550a5c-3cb1-4b06-a0cf-c71422ed4271 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '.version_select at 0x7f47f9316670>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:07.594288 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-e1550a5c-3cb1-4b06-a0cf-c71422ed4271 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-768906288" status: 202 len: 0 microversion: 2.1 time: 0.084505 May 23 16:24:07.594884 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2195/4269] 10.222.0.32 () {58 vars in 1217 bytes} [Mon May 23 16:24:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-768906288 => generated 0 bytes in 86 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:24:10.886498 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:10.887030 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:10.894521 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:10.895016 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:10.895831 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:10.896265 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:10.910569 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d HTTP/1.1 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:10.912132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.913961 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '1ad71e05-689f-42a2-8cf7-82c2a3562e9d'} May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:10.914909 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.916595 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:10.918392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 404 to user: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:10.918392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-2c4d66d9-4a5a-4daa-a9e2-c837788f4848 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 404 len: 111 microversion: 2.1 time: 0.030894 May 23 16:24:10.918392 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2075/4270] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:10 2022] DELETE /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:10.922641 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:10.923291 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 1ad71e05-689f-42a2-8cf7-82c2a3562e9d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:10.929914 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:10.930376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:10.931125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:10.931557 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:10.944701 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d HTTP/1.1 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:10.946310 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.948102 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '1ad71e05-689f-42a2-8cf7-82c2a3562e9d'} May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:10.949174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. May 23 16:24:10.950879 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:10.952560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 404 to user: Instance 1ad71e05-689f-42a2-8cf7-82c2a3562e9d could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:10.952560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-2a971b3b-7a78-4c9c-af98-e15d74b12d34 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d" status: 404 len: 111 microversion: 2.1 time: 0.029505 May 23 16:24:10.952560 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2196/4271] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:10 2022] GET /compute/v2.1/servers/1ad71e05-689f-42a2-8cf7-82c2a3562e9d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:12.204484 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:12.205014 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:12.211609 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:12.212065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:12.212825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:12.213267 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:12.226866 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 HTTP/1.1 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:12.228417 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.229980 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2'} May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:12.231387 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.232824 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:12.235177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 404 to user: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:12.235177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-dc641252-0e7f-4277-9a20-aa9be4b252d3 tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 404 len: 111 microversion: 2.1 time: 0.029519 May 23 16:24:12.235177 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2076/4272] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:12 2022] DELETE /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:12.238871 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:12.239534 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] [instance: 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:12.246059 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:12.246515 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:12.247344 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:12.247785 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:12.260938 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] HTTP exception thrown: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Request method failure captured: May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 HTTP/1.1 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6xdOSpbwWN1GBG0q4kGkoq5fHT44Yr6fOh41iI5fTTlyoaC5j8slh4FzgE_16cCFSh2HwnZEZhVykUY2R0oiOkUHF_ob45x5AicaAOd9gMKPqvuCFMJwZjTIR0B9Yh03EJUxY7-bZ6SMjWRNMVT4tc0KpjUwh6mvj3pmjUDbCOyUkU May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:12.262443 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/e80aa9835e9845e798b1c8346de141e3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_e80aa9835e9845e798b1c8346de141e3", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: e80aa9835e9845e798b1c8346de141e3 May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1441032771 May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 8b0a82159c6343d28546ec7c775b8791 May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeTestJSON-1441032771-project May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.264239 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '0d808c02-6a3e-4e38-bfd1-986e31e8cbd2'} May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:12.265503 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. May 23 16:24:12.267194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:12.268793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] Returning 404 to user: Instance 0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:12.268793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8360d8e8-0e24-449f-81a0-c58a9a412f9e tempest-AttachVolumeTestJSON-1441032771 tempest-AttachVolumeTestJSON-1441032771-project] 10.222.0.32 "GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2" status: 404 len: 111 microversion: 2.1 time: 0.029360 May 23 16:24:12.268793 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2197/4273] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:12 2022] GET /compute/v2.1/servers/0d808c02-6a3e-4e38-bfd1-986e31e8cbd2 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 16:24:26.376003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=7 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:26.376730 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=7 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:26.550494 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:26.551035 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:26.557816 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:26.558269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:26.559028 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:26.559485 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:26.582455 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-36e37de3-c07b-4893-883c-b3e90092a0f3 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/ips" status: 200 len: 150 microversion: 2.1 time: 0.211359 May 23 16:24:26.582998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2077/4274] 10.222.0.32 () {58 vars in 1247 bytes} [Mon May 23 16:24:26 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/ips => generated 150 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:29.261175 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "032cf841-7888-4bb9-8f7e-4ac4e5a6dfb2"}} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:24:29.262237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:29.264820 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=23,nova_api:DELETE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:29.265460 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=54,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:29.270174 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:29.270616 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:29.271390 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:29.271865 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:29.977418 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-687b7303-25a2-4e57-823d-0c7f5d43904e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "POST /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.720609 May 23 16:24:29.978028 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2198/4275] 10.222.0.32 () {60 vars in 1322 bytes} [Mon May 23 16:24:29 2022] POST /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments => generated 194 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:38.637573 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:38.638233 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:38.641482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=2 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:38.642148 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=4 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:24:38.650781 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:38.667616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:38.667616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:38.667616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:38.732380 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lazy-loading 'services' on Instance uuid 53650270-86f7-485b-931c-754469e60288 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:24:38.921051 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-def08fe5-5493-4dbe-9c7c-4d5b0773958b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments/032cf841-7888-4bb9-8f7e-4ac4e5a6dfb2" status: 202 len: 0 microversion: 2.1 time: 0.287074 May 23 16:24:38.921667 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2078/4276] 10.222.0.32 () {58 vars in 1415 bytes} [Mon May 23 16:24:38 2022] DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments/032cf841-7888-4bb9-8f7e-4ac4e5a6dfb2 => generated 0 bytes in 289 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:24:38.931355 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:38.932394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:38.939075 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:38.939496 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:38.940254 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:38.940683 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:38.969195 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8ffcce59-3b79-4ec4-8b00-ae5393e6160b tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042794 May 23 16:24:38.969751 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2199/4277] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:24:38 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:39.982343 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:39.983428 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:39.989550 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:39.989994 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:39.990736 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:39.991308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:40.022239 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3063403a-be02-4947-bd83-a2201294a878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045075 May 23 16:24:40.022787 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2079/4278] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:24:39 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:41.034430 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:41.035501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:41.042395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.042847 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.043644 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.044083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.073206 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b1c3eab2-1ecd-4da2-a080-0bdf7caba3d8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043230 May 23 16:24:41.073757 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2200/4279] 10.222.0.32 () {58 vars in 1301 bytes} [Mon May 23 16:24:41 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 23 16:24:41.131649 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:41.132164 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:41.139201 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.139650 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.140403 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.140835 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.169081 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Going to try to terminate instance {{(pid=124967) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:24:41.467287 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-1fed4a68-59b4-4a29-9840-7a78e19b96a8 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 204 len: 0 microversion: 2.1 time: 0.339190 May 23 16:24:41.468224 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2080/4280] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:41 2022] DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:24:41.476564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:41.477173 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:41.483807 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.484247 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.485038 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:41.485420 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:41.616376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:41.624625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-cf9cc7eb-fc0b-411c-bd58-9c521417d728 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1683 microversion: 2.1 time: 0.152075 May 23 16:24:41.625220 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2201/4281] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:41 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1683 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:42.641605 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:42.642304 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:42.648878 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:42.649327 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:42.650616 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:42.651049 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:42.784860 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:42.793230 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-779afd75-f6f1-4e68-8656-44dd2997dcd6 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1683 microversion: 2.1 time: 0.155764 May 23 16:24:42.793802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2081/4282] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:42 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1683 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:43.809894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:43.810530 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:43.817073 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:43.817524 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:43.818284 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:43.818718 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:44.060020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:44.068051 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-6936275c-b681-43f0-b2e4-55e3c6f28984 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1683 microversion: 2.1 time: 0.262140 May 23 16:24:44.072432 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2202/4283] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:43 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1683 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:45.040449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-57c553a1-d6c6-49b9-89b6-64f53fa57605 req-ef04a7c8-431f-4c19-860b-c270ebb98246 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "53650270-86f7-485b-931c-754469e60288", "name": "network-vif-deleted", "tag": "9860198a-b597-499a-b2ce-fe716df5978e"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:24:45.050457 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-57c553a1-d6c6-49b9-89b6-64f53fa57605 req-ef04a7c8-431f-4c19-860b-c270ebb98246 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:45.050968 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-57c553a1-d6c6-49b9-89b6-64f53fa57605 req-ef04a7c8-431f-4c19-860b-c270ebb98246 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:45.062869 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-57c553a1-d6c6-49b9-89b6-64f53fa57605 req-ef04a7c8-431f-4c19-860b-c270ebb98246 service nova] Creating event network-vif-deleted:9860198a-b597-499a-b2ce-fe716df5978e for instance 53650270-86f7-485b-931c-754469e60288 on hv-cinder-83588 May 23 16:24:45.068258 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-57c553a1-d6c6-49b9-89b6-64f53fa57605 req-ef04a7c8-431f-4c19-860b-c270ebb98246 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031639 May 23 16:24:45.068977 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2082/4284] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:24:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:24:45.086324 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:45.087083 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:45.094606 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:45.095178 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:45.102764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:45.102764 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:45.164109 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '72deeff329a6482cbfdb828a1996b127', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f474f456861e40ee9eb654852f80f313', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:45.176394 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-83d1706f-d8e1-4daa-aa26-d2bbf403c5dd tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 200 len: 1615 microversion: 2.1 time: 0.093881 May 23 16:24:45.177630 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2203/4285] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:45 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 1615 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:46.194776 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:46.195515 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:46.202237 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:46.202693 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:46.203552 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:46.203997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:46.217635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 HTTP/1.1 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:46.219274 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:46.221193 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '53650270-86f7-485b-931c-754469e60288'} May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:46.222260 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:46.223796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:46.225269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:46.225269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-c157d025-5e3e-4990-a93f-b5f4f4491878 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 404 len: 111 microversion: 2.1 time: 0.030613 May 23 16:24:46.225269 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2083/4286] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:46 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:46.229540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-78d9283f-3789-4819-92e9-ebb70da3a116 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '.version_select at 0x7f47feb07c10>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:46.342528 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-78d9283f-3789-4819-92e9-ebb70da3a116 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1293119602" status: 202 len: 0 microversion: 2.1 time: 0.116795 May 23 16:24:46.343116 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2204/4287] 10.222.0.32 () {58 vars in 1220 bytes} [Mon May 23 16:24:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1293119602 => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 23 16:24:48.299804 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:48.300309 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:48.307506 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.307958 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.308712 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.309149 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.322120 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 HTTP/1.1 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:48.323662 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.325412 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '53650270-86f7-485b-931c-754469e60288'} May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:48.326407 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.328065 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:48.329796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:48.329796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-f785bfb2-4771-4a03-922e-6c0f5a5e6b2e tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 404 len: 111 microversion: 2.1 time: 0.029102 May 23 16:24:48.329796 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2084/4288] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:48 2022] DELETE /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:48.333552 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:48.334167 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 53650270-86f7-485b-931c-754469e60288] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:48.341210 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.341662 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.342424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.342859 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.356020 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 HTTP/1.1 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:48.357669 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.359429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '53650270-86f7-485b-931c-754469e60288'} May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:48.360545 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. May 23 16:24:48.362237 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:48.366204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 53650270-86f7-485b-931c-754469e60288 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:48.366204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-8d629840-4624-4be4-b98a-ffc8dfa61bc7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288" status: 404 len: 111 microversion: 2.1 time: 0.029658 May 23 16:24:48.366204 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2205/4289] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:48 2022] GET /compute/v2.1/servers/53650270-86f7-485b-931c-754469e60288 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:48.903898 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:48.904825 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:48.911182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.911638 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.912396 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:48.912834 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:48.941007 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a9be30db-9a6f-4fe9-9013-54a054c89773 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.041992 May 23 16:24:48.941556 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2085/4290] 10.222.0.32 () {60 vars in 1342 bytes} [Mon May 23 16:24:48 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:24:48.998483 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:48.999006 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.005141 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.005646 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.006412 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.006853 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.032923 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:24:49.348385 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-266d6430-d608-464b-a872-17c76d1f477e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 204 len: 0 microversion: 2.20 time: 0.353462 May 23 16:24:49.349156 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2206/4291] 10.222.0.32 () {60 vars in 1279 bytes} [Mon May 23 16:24:48 2022] DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 16:24:49.357098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:49.357719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.364168 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.364618 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.365375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.365809 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.494986 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:49.503870 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:49.505389 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:49.506553 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-7481399c-f6d3-44ef-8cc9-994abd515113 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.153359 May 23 16:24:49.507132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2086/4292] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:49 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:24:49.584735 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:49.585312 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.591339 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.591788 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.592558 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.592976 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.606725 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd HTTP/1.1 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:49.608297 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.610009 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd'} May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:49.611042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.612694 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.614583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:49.614583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-ce3fe500-c6cf-482f-8de6-5911a1dce8df tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 404 len: 111 microversion: 2.1 time: 0.029109 May 23 16:24:49.614583 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2207/4293] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:49 2022] DELETE /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:49.618031 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:49.619321 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.625259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.625707 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.626464 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.626902 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.639777 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd HTTP/1.1 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:49.641314 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.643145 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd'} May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:49.644178 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. May 23 16:24:49.645829 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.647339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:49.647339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-840b16d6-404d-4225-884b-4dc7e1dc5c7d tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd" status: 404 len: 111 microversion: 2.1 time: 0.028465 May 23 16:24:49.647339 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2087/4294] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:49 2022] GET /compute/v2.1/servers/64ed6e5d-4adf-4d3d-96eb-bc8767d90bcd => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 16:24:49.650897 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:49.651429 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.657300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.657741 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.658745 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.659207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.672643 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a HTTP/1.1 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:49.674138 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.929845 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': 'c7915105-453d-453a-ad3c-656139c27d5a'} May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:49.930488 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.932216 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:49.933287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:49.933287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-27ea7946-0fc9-4bc2-b187-dc35caf540c7 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 404 len: 111 microversion: 2.1 time: 0.028337 May 23 16:24:49.933287 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2208/4295] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:49 2022] DELETE /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 16:24:49.933541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:49.933541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: c7915105-453d-453a-ad3c-656139c27d5a] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:49.933541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.933541 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a HTTP/1.1 May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:49.933791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:49.934600 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': 'c7915105-453d-453a-ad3c-656139c27d5a'} May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.935390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:49.936939 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance c7915105-453d-453a-ad3c-656139c27d5a could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-8579ab87-58dd-4c37-b895-56daceea08fe tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a" status: 404 len: 111 microversion: 2.1 time: 0.028095 May 23 16:24:49.937949 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2088/4296] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:49 2022] GET /compute/v2.1/servers/c7915105-453d-453a-ad3c-656139c27d5a => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:50.525760 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:50.526300 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:50.532799 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:50.533252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:50.534270 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:50.534709 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:50.665376 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:50.673834 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:50.675396 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:50.676581 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-255fde90-a7c7-427c-987c-6b8025323a65 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.154841 May 23 16:24:50.677146 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2209/4297] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:50 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:24:51.093173 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:51.093792 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:51.100211 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.100663 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.101419 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.101854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.119273 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c HTTP/1.1 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:51.123537 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.124488 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '4c5a0e23-b645-46dd-8924-37f041a76f7c'} May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:51.125098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.126095 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:51.126997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:51.126997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-4a7fb852-0ed7-4ced-b145-51d555eccf37 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 404 len: 111 microversion: 2.1 time: 0.036206 May 23 16:24:51.126997 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2089/4298] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:24:51 2022] DELETE /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:24:51.133518 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:51.134131 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] [instance: 4c5a0e23-b645-46dd-8924-37f041a76f7c] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:51.140707 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.141160 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.141919 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.142362 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.156433 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] HTTP exception thrown: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Request method failure captured: May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c HTTP/1.1 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v-UTK9_eHzkeNZQh7-RLPYOCg8ZLboY1giDzC4fgBNPXhw1XmSapoYQTfBBdPhA40WOlb7fyoMzrLWgZDIFIpYPtd4ZLuhrKwyGQTSUYlOX8wngYbj9rNup-4Ew4iBtZf52Eb2I6kqJyMiT9WCZN_W_EbS7t-1THN6OOveeUXq_Hc May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:51.157962 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/f474f456861e40ee9eb654852f80f313"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/f474f456861e40ee9eb654852f80f313"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_f474f456861e40ee9eb654852f80f313", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: f474f456861e40ee9eb654852f80f313 May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1048617199 May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 72deeff329a6482cbfdb828a1996b127 May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeNegativeTest-1048617199-project May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.160135 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '4c5a0e23-b645-46dd-8924-37f041a76f7c'} May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:51.164122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. May 23 16:24:51.165780 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:51.166567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] Returning 404 to user: Instance 4c5a0e23-b645-46dd-8924-37f041a76f7c could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:51.166567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d12a0671-d562-4f32-9b3f-8eae30a6a947 tempest-AttachVolumeNegativeTest-1048617199 tempest-AttachVolumeNegativeTest-1048617199-project] 10.222.0.32 "GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c" status: 404 len: 111 microversion: 2.1 time: 0.030032 May 23 16:24:51.166567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2210/4299] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:24:51 2022] GET /compute/v2.1/servers/4c5a0e23-b645-46dd-8924-37f041a76f7c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 23 16:24:51.694992 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:51.695562 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:51.702673 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.703132 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.703915 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:51.704346 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:51.904613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:51.913249 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:51.914889 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:51.916182 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e432a4ba-ced9-49ba-931c-37be029c5bf2 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1721 microversion: 2.20 time: 0.225420 May 23 16:24:51.916806 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2090/4300] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:51 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1721 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:24:52.381569 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [req-5e4059a9-640d-4fb6-b3d4-30151c92b942 req-57ff207e-226d-4e10-8f0c-f9d973074bd4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8920fd46-7732-48a4-a860-831350e77715", "name": "network-vif-deleted", "tag": "25c892d0-d9ab-40be-b315-ecff2dafaea5"}]} {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:24:52.390081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5e4059a9-640d-4fb6-b3d4-30151c92b942 req-57ff207e-226d-4e10-8f0c-f9d973074bd4 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:52.390564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [req-5e4059a9-640d-4fb6-b3d4-30151c92b942 req-57ff207e-226d-4e10-8f0c-f9d973074bd4 service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:52.401522 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.compute.server_external_events [req-5e4059a9-640d-4fb6-b3d4-30151c92b942 req-57ff207e-226d-4e10-8f0c-f9d973074bd4 service nova] Creating event network-vif-deleted:25c892d0-d9ab-40be-b315-ecff2dafaea5 for instance 8920fd46-7732-48a4-a860-831350e77715 on hv-cinder-83588 May 23 16:24:52.407555 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [req-5e4059a9-640d-4fb6-b3d4-30151c92b942 req-57ff207e-226d-4e10-8f0c-f9d973074bd4 service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029588 May 23 16:24:52.415826 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2211/4301] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:24:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 23 16:24:52.935272 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:52.936036 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:52.942655 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:52.943150 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:52.944101 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:52.944531 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:52.979124 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:52.979801 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:52.980263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.objects.instance [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lazy-loading 'fault' on Instance uuid 8920fd46-7732-48a4-a860-831350e77715 {{(pid=124967) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:24:53.025003 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:53.033190 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:53.035131 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6d689e7ca175463ab0c319260ba49d0d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7651f704e84b4ce8bbd53a6292126bce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:24:53.039194 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-678354a3-8602-4e48-9c3f-dc9d5c1c31eb tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 200 len: 1371 microversion: 2.20 time: 0.108588 May 23 16:24:53.051658 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2091/4302] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:52 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 1371 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 23 16:24:54.058910 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:54.059647 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:54.065882 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:54.066326 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:54.067081 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:54.067716 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:54.081800 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] HTTP exception thrown: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Request method failure captured: May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 HTTP/1.1 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v_K5SJuLxj5xaYmnKSQGdX8-ZyokPkO50EU6mopqWihiAGV-JLe0Uo6AsG3W9pC-Pa9sBfyQiH17x3KqkB0w_rWURS826NfEEYUYyAK0DaL_khCz1GDkABhQCEoB-ChOsjVXyqOa7RMKZkvAJxr19IY9xTg1gp72ykaKNqq8AEL_s May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Openstack-Nova-Api-Version: 2.20 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:54.083479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7651f704e84b4ce8bbd53a6292126bce", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6d689e7ca175463ab0c319260ba49d0d May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:54.084729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '8920fd46-7732-48a4-a860-831350e77715'} May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:54.085479 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:54.086436 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:54.087452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Returning 404 to user: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:54.087452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-f544dcaa-63ae-4b04-84d5-4901b915b741 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 404 len: 111 microversion: 2.20 time: 0.030771 May 23 16:24:54.087452 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2212/4303] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:54 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 16:24:54.094482 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-d46791c3-7e8f-430b-9529-34f0ce4ace96 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '.version_select at 0x7f47fea07af0>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:54.165513 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-d46791c3-7e8f-430b-9529-34f0ce4ace96 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-113322561" status: 204 len: 0 microversion: 2.20 time: 0.074798 May 23 16:24:54.166099 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2092/4304] 10.222.0.32 () {60 vars in 1258 bytes} [Mon May 23 16:24:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-113322561 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) May 23 16:24:56.454301 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:56.454889 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:56.461440 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:56.465057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:56.465057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:56.465057 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:56.482389 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] HTTP exception thrown: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Request method failure captured: May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 HTTP/1.1 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v_K5SJuLxj5xaYmnKSQGdX8-ZyokPkO50EU6mopqWihiAGV-JLe0Uo6AsG3W9pC-Pa9sBfyQiH17x3KqkB0w_rWURS826NfEEYUYyAK0DaL_khCz1GDkABhQCEoB-ChOsjVXyqOa7RMKZkvAJxr19IY9xTg1gp72ykaKNqq8AEL_s May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Openstack-Nova-Api-Version: 2.20 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:56.484502 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7651f704e84b4ce8bbd53a6292126bce", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6d689e7ca175463ab0c319260ba49d0d May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.485637 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '8920fd46-7732-48a4-a860-831350e77715'} May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:56.486328 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.488246 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:56.489931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Returning 404 to user: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:56.489931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-fad4efd7-0583-4e10-971f-82f63a2c6899 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 404 len: 111 microversion: 2.20 time: 0.037150 May 23 16:24:56.489931 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2213/4305] 10.222.0.32 () {60 vars in 1279 bytes} [Mon May 23 16:24:56 2022] DELETE /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 16:24:56.497181 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:56.497839 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 8920fd46-7732-48a4-a860-831350e77715] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:56.505084 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:56.505542 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:56.506308 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:56.506750 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:56.521302 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] HTTP exception thrown: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Request method failure captured: May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 HTTP/1.1 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v_K5SJuLxj5xaYmnKSQGdX8-ZyokPkO50EU6mopqWihiAGV-JLe0Uo6AsG3W9pC-Pa9sBfyQiH17x3KqkB0w_rWURS826NfEEYUYyAK0DaL_khCz1GDkABhQCEoB-ChOsjVXyqOa7RMKZkvAJxr19IY9xTg1gp72ykaKNqq8AEL_s May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Openstack-Nova-Api-Version: 2.20 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:56.522998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7651f704e84b4ce8bbd53a6292126bce", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 6d689e7ca175463ab0c319260ba49d0d May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.524374 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '8920fd46-7732-48a4-a860-831350e77715'} May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:56.525098 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. May 23 16:24:56.525998 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:56.526922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Returning 404 to user: Instance 8920fd46-7732-48a4-a860-831350e77715 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:56.526922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-a2969373-f050-4133-bd27-a3ea650e523e tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715" status: 404 len: 111 microversion: 2.20 time: 0.032300 May 23 16:24:56.526922 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2093/4306] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:56 2022] GET /compute/v2.1/servers/8920fd46-7732-48a4-a860-831350e77715 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 23 16:24:57.762278 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:57.763458 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:57.769697 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:57.770147 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:57.770901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:57.771364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:57.785369 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.wsgi [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] HTTP exception thrown: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Request method failure captured: May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: request: DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d HTTP/1.1 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept: application/json May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Accept-Encoding: identity May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Connection: close May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Length: 0 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Content-Type: application/json May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Host: 10.222.0.32 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Openstack-System-Scope: None May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: User-Agent: python-urllib3/1.26.9 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Auth-Token: gAAAAABii6v_K5SJuLxj5xaYmnKSQGdX8-ZyokPkO50EU6mopqWihiAGV-JLe0Uo6AsG3W9pC-Pa9sBfyQiH17x3KqkB0w_rWURS826NfEEYUYyAK0DaL_khCz1GDkABhQCEoB-ChOsjVXyqOa7RMKZkvAJxr19IY9xTg1gp72ykaKNqq8AEL_s May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Id: None May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Domain-Name: None May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Identity-Status: Confirmed May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Is-Admin-Project: True May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Openstack-Nova-Api-Version: 2.20 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Id: default May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Domain-Name: Default May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Role: member,reader May 23 16:24:57.786908 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Roles: member,reader May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7651f704e84b4ce8bbd53a6292126bce", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Id: default May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Domain-Name: Default May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Id: 6d689e7ca175463ab0c319260ba49d0d May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: method: > May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: exception: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.788252 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: action_args: {'id': '6af85f59-fe73-4757-ab1a-d0ad26e6bd0d'} May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:57.788946 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.793642 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR nova.api.openstack.wsgi May 23 16:24:57.794893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Returning 404 to user: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. {{(pid=124966) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:57.794893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-af79bc3c-73fa-44d9-887c-af3ef11e6cd8 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 404 len: 111 microversion: 2.20 time: 0.030681 May 23 16:24:57.794893 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2214/4307] 10.222.0.32 () {60 vars in 1279 bytes} [Mon May 23 16:24:57 2022] DELETE /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 23 16:24:57.797067 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:24:57.797686 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] [instance: 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:24:57.803941 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:57.804390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:57.805140 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:24:57.805635 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:24:57.818742 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] HTTP exception thrown: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Request method failure captured: May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d HTTP/1.1 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6v_K5SJuLxj5xaYmnKSQGdX8-ZyokPkO50EU6mopqWihiAGV-JLe0Uo6AsG3W9pC-Pa9sBfyQiH17x3KqkB0w_rWURS826NfEEYUYyAK0DaL_khCz1GDkABhQCEoB-ChOsjVXyqOa7RMKZkvAJxr19IY9xTg1gp72ykaKNqq8AEL_s May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Openstack-Nova-Api-Version: 2.20 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:24:57.820340 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/7651f704e84b4ce8bbd53a6292126bce"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_7651f704e84b4ce8bbd53a6292126bce", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: 7651f704e84b4ce8bbd53a6292126bce May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1690800084 May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 6d689e7ca175463ab0c319260ba49d0d May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1690800084-project May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.821692 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '6af85f59-fe73-4757-ab1a-d0ad26e6bd0d'} May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:24:57.822336 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. May 23 16:24:57.823476 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:24:57.824558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] Returning 404 to user: Instance 6af85f59-fe73-4757-ab1a-d0ad26e6bd0d could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:24:57.824558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-3312ab4c-32fa-45bf-b81b-850879337a18 tempest-AttachVolumeShelveTestJSON-1690800084 tempest-AttachVolumeShelveTestJSON-1690800084-project] 10.222.0.32 "GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d" status: 404 len: 111 microversion: 2.20 time: 0.028458 May 23 16:24:57.824558 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2094/4308] 10.222.0.32 () {60 vars in 1276 bytes} [Mon May 23 16:24:57 2022] GET /compute/v2.1/servers/6af85f59-fe73-4757-ab1a-d0ad26e6bd0d => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 23 16:26:56.029202 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:26:56.029773 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:26:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:27:06.781222 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:27:06.781701 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:27:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:28:18.040768 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:28:18.041439 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:28:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:28:30.438759 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: ERROR oslo.messaging._drivers.impl_rabbit [-] [dd197da5-4f33-41b6-86a1-1b3ce2220c6a] AMQP server on 10.222.0.32:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 23 16:28:30.439640 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_api:SELECT=40,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:28:30.440366 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG dbcounter [-] [124966] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124966) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:28:30.444880 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:30.445367 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:30.451125 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:30.451584 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:30.452336 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:30.452763 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:30.477501 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:28:30.767512 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-d0b27109-5a92-439c-b559-a8fb6ac51c9c tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 204 len: 0 microversion: 2.1 time: 0.330244 May 23 16:28:30.768096 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2215/4309] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:28:30 2022] DELETE /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:28:30.774423 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR oslo.messaging._drivers.impl_rabbit [-] [3056c5c2-02e1-4db6-8e90-5f7b8a22565e] AMQP server on 10.222.0.32:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 23 16:28:30.775283 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:28:30.776011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG dbcounter [-] [124967] Writing DB stats nova_cell1:SELECT=77,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124967) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:28:30.780761 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:30.781948 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:30.789167 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:30.789617 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:30.790369 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:30.790802 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:31.031652 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:31.039613 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-e3e95636-8cea-4df9-9742-ddc335e2b156 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1507 microversion: 2.1 time: 0.266949 May 23 16:28:31.040165 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2095/4310] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:30 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1507 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:32.056484 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:32.057194 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:32.064901 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:32.065352 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:32.066122 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:32.066567 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:32.073203 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] [dd197da5-4f33-41b6-86a1-1b3ce2220c6a] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 33224. May 23 16:28:32.325159 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:32.332588 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-a45fb91b-2137-40fd-a3a1-daab74d99c3b tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1507 microversion: 2.1 time: 0.280342 May 23 16:28:32.338349 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2216/4311] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:32 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1507 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:32.526674 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] [3056c5c2-02e1-4db6-8e90-5f7b8a22565e] Reconnected to AMQP server on 10.222.0.32:5672 via [amqp] client with port 33234. May 23 16:28:32.578965 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-02fbe743-f705-4025-8fe6-335e5f726f7d req-97cd767a-4c5b-4cb7-9f48-609f104dfccd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6", "name": "network-vif-deleted", "tag": "ee897199-ccbf-4d09-971b-4358bcee64ed"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:28:32.589076 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-02fbe743-f705-4025-8fe6-335e5f726f7d req-97cd767a-4c5b-4cb7-9f48-609f104dfccd service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:32.589520 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-02fbe743-f705-4025-8fe6-335e5f726f7d req-97cd767a-4c5b-4cb7-9f48-609f104dfccd service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:32.601549 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-02fbe743-f705-4025-8fe6-335e5f726f7d req-97cd767a-4c5b-4cb7-9f48-609f104dfccd service nova] Creating event network-vif-deleted:ee897199-ccbf-4d09-971b-4358bcee64ed for instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 on hv-cinder-83588 May 23 16:28:32.609539 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-02fbe743-f705-4025-8fe6-335e5f726f7d req-97cd767a-4c5b-4cb7-9f48-609f104dfccd service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097330 May 23 16:28:32.610135 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2096/4312] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:28:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:28:33.350261 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:33.350873 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:33.357364 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:33.357813 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:33.358564 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:33.358992 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:33.389762 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:33.390207 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:33.390655 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'fault' on Instance uuid 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:28:33.431463 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:33.439039 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-961c49f8-3f71-43f0-82a8-f3e353fad0ef tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 200 len: 1311 microversion: 2.1 time: 0.093210 May 23 16:28:33.439625 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2217/4313] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:33 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 1311 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:34.457180 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:34.457872 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:34.464631 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.465083 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:34.465828 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.466263 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:34.479871 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 HTTP/1.1 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:28:34.481449 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. May 23 16:28:34.482850 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6'} May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:28:34.483719 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. May 23 16:28:34.485055 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:34.486212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 404 to user: Instance 058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:28:34.486212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-89523544-b949-4236-8336-c9dd8c6b0414 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6" status: 404 len: 111 microversion: 2.1 time: 0.030604 May 23 16:28:34.486212 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2097/4314] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:34 2022] GET /compute/v2.1/servers/058bc695-1e1a-4a2c-a7c9-1c78a1e33ec6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:28:34.491814 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:34.492313 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:34.498734 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.499227 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:34.499985 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.500424 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:34.524507 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Going to try to terminate instance {{(pid=124966) delete /opt/stack/nova/nova/compute/api.py:2664}} May 23 16:28:34.836181 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-b26c3350-54e9-4dca-abc9-6836497940c8 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "DELETE /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 204 len: 0 microversion: 2.1 time: 0.348517 May 23 16:28:34.836728 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2218/4315] 10.222.0.32 () {58 vars in 1238 bytes} [Mon May 23 16:28:34 2022] DELETE /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 23 16:28:34.984362 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:34.985125 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:34.991227 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.991685 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:34.992445 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:34.992888 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:35.123799 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.policy [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124967) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:35.131791 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-263d9613-8f1d-482d-8024-0634eebbe0c3 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.290273 May 23 16:28:35.132364 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2098/4316] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:34 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:36.149190 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:36.149924 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:36.156256 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:36.156704 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:36.157462 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:36.157894 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:36.369926 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:36.377540 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-21008e9b-ebd4-4a93-a474-dc43dff09303 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1521 microversion: 2.1 time: 0.232549 May 23 16:28:36.378238 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2219/4317] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:36 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1521 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:36.663995 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [req-997f3734-1b8f-48a4-a8ed-e79e54b60837 req-0ff0f586-a181-4777-a0c1-aee3936eb3eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3831639c-bfc1-491d-a152-5b09cf9adf26", "name": "network-vif-deleted", "tag": "2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8"}]} {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 23 16:28:36.672298 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-997f3734-1b8f-48a4-a8ed-e79e54b60837 req-0ff0f586-a181-4777-a0c1-aee3936eb3eb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:36.672733 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [req-997f3734-1b8f-48a4-a8ed-e79e54b60837 req-0ff0f586-a181-4777-a0c1-aee3936eb3eb service nova] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:36.684011 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.compute.server_external_events [req-997f3734-1b8f-48a4-a8ed-e79e54b60837 req-0ff0f586-a181-4777-a0c1-aee3936eb3eb service nova] Creating event network-vif-deleted:2bc190cc-24a1-4a7a-9ed2-77bdae8fb3d8 for instance 3831639c-bfc1-491d-a152-5b09cf9adf26 on hv-cinder-83588 May 23 16:28:36.689375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [req-997f3734-1b8f-48a4-a8ed-e79e54b60837 req-0ff0f586-a181-4777-a0c1-aee3936eb3eb service nova] 10.222.0.32 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029785 May 23 16:28:36.689985 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2099/4318] 10.222.0.32 () {64 vars in 1317 bytes} [Mon May 23 16:28:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 23 16:28:37.394395 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.api.openstack.wsgi [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124966) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:37.395007 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.compute.api [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124966) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:37.402072 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:37.402516 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:37.403292 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:37.403729 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:37.432866 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:37.433303 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG oslo_concurrency.lockutils [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124966) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:37.433744 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.objects.instance [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lazy-loading 'fault' on Instance uuid 3831639c-bfc1-491d-a152-5b09cf9adf26 {{(pid=124966) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 23 16:28:37.475386 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: DEBUG nova.policy [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '606ff49786a04aa5b677cbb9a57b216c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad4a74e93cb54747a64778b2dbea9da8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124966) authorize /opt/stack/nova/nova/policy.py:199}} May 23 16:28:37.482714 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO nova.api.openstack.requestlog [None req-c6002d9b-dc12-481a-bb3e-67c35801e24f tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 200 len: 1311 microversion: 2.1 time: 0.092159 May 23 16:28:37.483295 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: [pid: 124966|app: 0|req: 2220/4319] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:37 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 1311 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 23 16:28:38.501297 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Calling method '>' {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 23 16:28:38.502060 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.compute.api [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] [instance: 3831639c-bfc1-491d-a152-5b09cf9adf26] Fetching instance by UUID {{(pid=124967) get /opt/stack/nova/nova/compute/api.py:2920}} May 23 16:28:38.508900 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:38.509352 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:38.510102 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 23 16:28:38.510535 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG oslo_concurrency.lockutils [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Lock "99fe71d5-840c-46a3-b7af-a004ed27ab79" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124967) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 23 16:28:38.523300 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.wsgi [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] HTTP exception thrown: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Request method failure captured: May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: request: GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 HTTP/1.1 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept: application/json May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Accept-Encoding: identity May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Connection: close May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Length: 0 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Content-Type: application/json May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Host: 10.222.0.32 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Openstack-System-Scope: None May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: User-Agent: python-urllib3/1.26.9 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Auth-Token: gAAAAABii6wEGN-oNwHxf4XZQLgi7VhumgjITrGEmdbCd8M1LmGkMlYHlSk48ANyPeUCKYCz8Imq7iGEl7ZNOaYbPC5ZvVassLDV5v7EanmcPS8KCfm1oj7z9GzRFqGo44xPDc1lutPJ3MhgugDi8HbWjOD48-zlZUoPRaQnPfZEwax0fwFqFsA May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Id: None May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Domain-Name: None May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Identity-Status: Confirmed May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Is-Admin-Project: True May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Id: default May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Domain-Name: Default May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Role: member,reader May 23 16:28:38.525375 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Roles: member,reader May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:9696/"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32/volume/v3/ad4a74e93cb54747a64778b2dbea9da8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.32:8080/v1/AUTH_ad4a74e93cb54747a64778b2dbea9da8", "adminURL": "http://10.222.0.32:8080"}]}] May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Id: ad4a74e93cb54747a64778b2dbea9da8 May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-287032477 May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Id: default May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Domain-Name: Default May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Id: 606ff49786a04aa5b677cbb9a57b216c May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: X-User-Name: tempest-ServerRescueNegativeTestJSON-287032477-project May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: method: > May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: exception: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. May 23 16:28:38.526684 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: action_args: {'id': '3831639c-bfc1-491d-a152-5b09cf9adf26'} May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: {{(pid=124967) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 23 16:28:38.527461 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. May 23 16:28:38.528749 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: ERROR nova.api.openstack.wsgi May 23 16:28:38.529883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: DEBUG nova.api.openstack.wsgi [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] Returning 404 to user: Instance 3831639c-bfc1-491d-a152-5b09cf9adf26 could not be found. {{(pid=124967) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 23 16:28:38.529883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO nova.api.openstack.requestlog [None req-40aba0bd-75a2-40c8-b37f-674d0f672e46 tempest-ServerRescueNegativeTestJSON-287032477 tempest-ServerRescueNegativeTestJSON-287032477-project] 10.222.0.32 "GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26" status: 404 len: 111 microversion: 2.1 time: 0.030249 May 23 16:28:38.529883 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: [pid: 124967|app: 0|req: 2100/4320] 10.222.0.32 () {58 vars in 1235 bytes} [Mon May 23 16:28:38 2022] GET /compute/v2.1/servers/3831639c-bfc1-491d-a152-5b09cf9adf26 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 23 16:30:03.984659 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:30:03.985235 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:30:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:30:04.661192 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:30:04.661738 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:30:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:30:16.037592 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:30:16.038056 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:30:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:31:38.048259 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:31:38.048884 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:31:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:33:23.992414 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:33:23.993042 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:33:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:33:24.668932 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:33:24.669509 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:33:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:33:36.046189 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:33:36.046767 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:33:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:33:38.828508 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:33:38.829134 dv-cinder-835881-14-iscsi devstack@n-api.service[124966]: Mon May 23 16:33:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:33:46.793004 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:33:46.793854 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:33:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 23 16:34:58.056805 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 23 16:34:58.057390 dv-cinder-835881-14-iscsi devstack@n-api.service[124967]: Mon May 23 16:34:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!